| DC 欄位 |
值 |
語言 |
| DC.contributor | 資訊工程學系 | zh_TW |
| DC.creator | 高聿緯 | zh_TW |
| DC.creator | Yu-Wei Kao | en_US |
| dc.date.accessioned | 2019-11-8T07:39:07Z | |
| dc.date.available | 2019-11-8T07:39:07Z | |
| dc.date.issued | 2019 | |
| dc.identifier.uri | http://ir.lib.ncu.edu.tw:88/thesis/view_etd.asp?URN=106522127 | |
| dc.contributor.department | 資訊工程學系 | zh_TW |
| DC.description | 國立中央大學 | zh_TW |
| DC.description | National Central University | en_US |
| dc.description.abstract | 倒傳遞演算法已被廣泛的運用在深度學習上，但因為有傳遞鎖與梯度
消失、爆炸的問題，它不是有效率且穩定的演算法，在較深的網路架構
更可以觀察到這些現象。此外，單單只用一個目標來更新神經網路中的
參數在生物學來說並非合理的。
在本篇論文中，我們提出了一種新穎且受生物學啟發的學習架構，名
為「關聯式學習」，這個訓練方式將原有的神經網路模組化成小單元，每
個小單元都有自己的局部目標，又因為這些單元兩兩獨立，關聯式學習
能夠獨立且同時訓練彼此獨立的參數。
令人驚訝的是，利用關聯式學習訓練的準確度，也能與直接使用目標
訓練的傳統倒傳遞演算法相當，此外，可能是因為模組內的梯度流較短，
關聯式學習也能訓練用sigmoid 當作活化函數的深度學習網路，然而若
是用倒傳遞演算法訓練這類網路會容易導致梯度消失。
我們也透過觀察隱藏層中的類間與類內距離，以及t-SNE 來呈
現數量上與品質上的結果，發現聯想式學習能夠生成更好的間特徵
(Metafeatures)。 | zh_TW |
| dc.description.abstract | Backpropagation has been widely used in deep learning approaches, but
it is inefficient and sometimes unstable because of backward locking and
vanishing/exploding gradient problems, especially when the gradient flow
is long. Additionally, updating all edge weights based on a single objective
seems biologically implausible. In this paper, we introduce a novel biologically
motivated learning structure called Associated Learning, which
modularizes the network into smaller components, each of which has a local
objective. Because the objectives are mutually independent, Associated
Learning can learn the parameters independently and simultaneously when
these parameters belong to different components. Surprisingly, training
deep models by Associated Learning yields comparable accuracies to models
trained using typical backpropagation methods, which aims at fitting
the target variable directly. Moreover, probably because the gradient flow
of each component is short, deep networks can still be trained with Associated
Learning even when some of the activation functions are sigmoid—a
situation that usually results in the vanishing gradient problem when using
typical backpropagation. We also found that the Associated Learning generates better metafeatures, which we demonstrated both quantitatively
(via inter-class and intra-class distance comparisons in the hidden layers)
and qualitatively (by visualizing the hidden layers using t-SNE). | en_US |
| DC.subject | 生物合理性演算法 | zh_TW |
| DC.subject | 深度學習 | zh_TW |
| DC.subject | 平行運算 | zh_TW |
| DC.subject | 模組化 | zh_TW |
| DC.subject | Biologically plausible algorithm | en_US |
| DC.subject | Deep learning | en_US |
| DC.subject | Parallel computing | en_US |
| DC.subject | Modularization | en_US |
| DC.title | 關聯式學習：利用自動編碼器與目標傳遞法分解端到端倒傳遞演算法 | zh_TW |
| dc.language.iso | zh-TW | zh-TW |
| DC.title | Associated Learning: Decomposing End-to-end Backpropagation based on Auto-encoders and Target Propagation | en_US |
| DC.type | 博碩士論文 | zh_TW |
| DC.type | thesis | en_US |
| DC.publisher | National Central University | en_US |