藉由權重之梯度大小調整DropConnect的捨棄機率來訓練神經網路;Training a neural network by adjusting the drop probability in DropConnect based on the magnitude of the gradient

NCU Institutional Repository > 資訊電機學院 > 資訊工程研究所 > 博碩士論文 > Item 987654321/89773

請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/89773

題名:	藉由權重之梯度大小調整DropConnect的捨棄機率來訓練神經網路;Training a neural network by adjusting the drop probability in DropConnect based on the magnitude of the gradient
作者:	楊緣智;Yang, Yuan-Chih
貢獻者:	資訊工程學系
關鍵詞:	過度擬合;正則化;Dropout;DropConnect;泛化;Overfitting;Regularization;Dropout;DropConnect;Generalization
日期:	2022-07-19
上傳時間:	2022-10-04 11:59:10 (UTC+8)
出版者:	國立中央大學
摘要:	在深度學習訓練中，Dropout 和 DropConnect 是常被用來解決過度擬合的正則化技術，Dropout 和 DropConnect 藉由在訓練過程中以一個固定機率隨機地捨棄神經元及該神經元前後的連結，使得每個神經元彼此之間不會過度依賴其他神經元，進而提高模型泛化的能力。本文提出了一種新模型 Gradient DropConnect，它利用每個權重和偏差的梯度以確定它們在訓練期間的下降捨棄機率。我們進行了一連串的實驗以驗證這種方法可以有效地緩解過度擬合。;Dropout and DropConnect are regularization techniques often used to address the overfitting issue in deep learning. Dropout and DropConnect randomly discard neurons or links with a fixed probability during training so that each neuron does not depend too much on other neurons, thereby improving the model’s generalization ability. This paper proposes a new model, Gradient DropConnect, which leverages the gradient of each weight and bias to determine their dropping probabilities during training. We conducted thorough experiments to validate that such an approach can effectively mitigate overfitting.
顯示於類別:	[資訊工程研究所] 博碩士論文

文件中的檔案:

檔案	描述	大小	格式	瀏覽次數
index.html		0Kb	HTML	25	檢視/開啟

在NCUIR中所有的資料項目都受到原著作權保護.

社群 sharing

資料載入中.....