倒傳遞 (Backpropagation, BP) 是當今深度神經網路更新權重演算法 的基石，但反向傳播因反向鎖定 (backward locking) 的問題而效率不佳。 本研究試圖解決反向鎖定問題，並將提出的新方法命名為 Supervised Contrastive Parallel Learning (SCPL)，SCPL 利用監督對比損失函數作為每個卷積層的區域目標函數，因為每一層的區域目標函數間互相隔離， SCPL 可以平行地學習不同卷基層的權重。 本論文亦和過去在神經網路平行化的研究進行比較，探討現存方法 各自的優勢與限制，並討論此議題未來的研究方向。;Backpropagation (BP) is the cornerstone of today’s deep learning al gorithms to update the weights in deep neural networks, but it is inefficient partially because of the backward locking problem. This thesis proposes Supervised Contrastive Parallel Learning (SCPL) to address the issue of backward locking. SCPL uses the supervised contrastive loss as the local objective function for each layer. Because the local objective functions in different layers are isolated, SCPL can learn the weights of different lay ers in parallel. We compare SCPL with recent works on neural network parallelization. We discuss the advantages and limitations of the existing methods. Finally, we suggest future research directions on neural network parallelization.
