藉由權重之梯度大小調整DropConnect的捨棄機率來訓練神經網路;Training a neural network by adjusting the drop probability in DropConnect based on the magnitude of the gradient

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/89773

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/89773

Title:	藉由權重之梯度大小調整DropConnect的捨棄機率來訓練神經網路;Training a neural network by adjusting the drop probability in DropConnect based on the magnitude of the gradient
Authors:	楊緣智;Yang, Yuan-Chih
Contributors:	資訊工程學系
Keywords:	過度擬合;正則化;Dropout;DropConnect;泛化;Overfitting;Regularization;Dropout;DropConnect;Generalization
Date:	2022-07-19
Issue Date:	2022-10-04 11:59:10 (UTC+8)
Publisher:	國立中央大學
Abstract:	在深度學習訓練中，Dropout 和 DropConnect 是常被用來解決過度擬合的正則化技術，Dropout 和 DropConnect 藉由在訓練過程中以一個固定機率隨機地捨棄神經元及該神經元前後的連結，使得每個神經元彼此之間不會過度依賴其他神經元，進而提高模型泛化的能力。本文提出了一種新模型 Gradient DropConnect，它利用每個權重和偏差的梯度以確定它們在訓練期間的下降捨棄機率。我們進行了一連串的實驗以驗證這種方法可以有效地緩解過度擬合。;Dropout and DropConnect are regularization techniques often used to address the overfitting issue in deep learning. Dropout and DropConnect randomly discard neurons or links with a fixed probability during training so that each neuron does not depend too much on other neurons, thereby improving the model’s generalization ability. This paper proposes a new model, Gradient DropConnect, which leverages the gradient of each weight and bias to determine their dropping probabilities during training. We conducted thorough experiments to validate that such an approach can effectively mitigate overfitting.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	51	View/Open

社群 sharing

Loading...