博碩士論文 106522067 詳細資訊




以作者查詢圖書館館藏 以作者查詢臺灣博碩士 以作者查詢全國書目 勘誤回報 、線上人數:79 、訪客IP:44.200.144.68
姓名 王莛瑜(Ting-Yu Wang)  查詢紙本館藏   畢業系所 資訊工程學系
論文名稱 基於類別不變性密集對抗式遷移學習
(Dense Adversarial Transfer Learning Based On Class-invariant)
相關論文
★ Single and Multi-Label Environmental Sound Recognition with Gaussian Process★ 波束形成與音訊前處理之嵌入式系統實現
★ 語音合成及語者轉換之應用與設計★ 基於語意之輿情分析系統
★ 高品質口述系統之設計與應用★ 深度學習及加速強健特徵之CT影像跟骨骨折辨識及偵測
★ 基於風格向量空間之個性化協同過濾服裝推薦系統★ RetinaNet應用於人臉偵測
★ 金融商品走勢預測★ 整合深度學習方法預測年齡以及衰老基因之研究
★ 漢語之端到端語音合成研究★ 基於 ARM 架構上的 ORB-SLAM2 的應用與改進
★ 基於深度學習之指數股票型基金趨勢預測★ 探討財經新聞與金融趨勢的相關性
★ 基於卷積神經網路的情緒語音分析★ 運用深度學習方法預測阿茲海默症惡化與腦中風手術存活
檔案 [Endnote RIS 格式]    [Bibtex 格式]    [相關文章]   [文章引用]   [完整記錄]   [館藏目錄]   至系統瀏覽論文 ( 永不開放)
摘要(中) 近年來深度卷積神經網路在影像辨識領域取得相當的成功,這樣的成功需要大量資料標籤作為訓練資料,然而現實場景中獲取標籤的成本卻是相當龐大的,於是推展了遷移學習的研究。
遷移學習為了解決訓練資料不足的問題,其假設訓練與測試資料為獨立同分布,目的將知識從來源域遷移至目標域,使得在目標任務資料集只有少量標籤資料甚至無標籤資料情況下,深度學習模型亦可透過現有標籤資料訓練學習,省下蒐集成本。近年來,逐漸發展出對抗式遷移學習應用於領域自適應的場景中,對抗式遷移學習主要透過額外的領域判別器與特徵擷取器進行最大最小化對抗學習,藉以抓取領域不變性特徵拉近領域分布距離達到領域自適應。
本論文採用CAN架構概念,其作為首個使用所有區塊特徵搭配領域判別器的高精度模型,CAN透過淺層區塊學習領域特徵,末層區塊透過反向梯度層學習領域不變性特徵。本論文提出之架構同樣使用各層區塊特徵增益遷移,透過權重式密集連接各區塊特徵並搭配反向梯度層,且在領域判別器中加入分類器,使其在學習領域不變性特徵時保有分類能力,增強領域自適應能力,最終於實驗中比較提出架構與CAN架構結果,於Office資料集中3個遷移任務中準確度明顯提升0.3~0.5。
摘要(英) In recent years, deep neural networks (DNN) have demonstrated their powerful learning ability in image recognition. Such success result requires a large-scale data set as training data. However, acquiring data label in real-life needs high cost. This dilemma condition makes the study of transfer learning.
In order to solve the problem of insufficient training data, transfer learning assumes that the training and test dataset are independent and identically distributed. Its purpose is to transfer knowledge from the source domain to the target domain. Thus, even if target task data set has only a small amount of label data or no label data, deep neural network still can be used to train and learn through existing label data.
Nowadays, adversarial-based deep transfer learning has been gradually applied in the domain adaption task. It uses a extra domain discriminator to learn domain-invariant features by minimizing the loss of domain discriminator and utilizing reversed gradient during the backpropagation process for aligning distributions.
The architecture proposed in this paper based on concept of collaborative and adversarial network(CAN),which is the first model using all of blocks features with high accuracy. It use all blocks with domain discriminators and let them learn domain feature except last one learning for domain-invariant feature by a reversal gradient layer. Our architecture also use all block features to advance transfer effect by weighted dense connection with a gradient reversal layer. In addition, our architecture add a classifier in domain discriminator, in order to make network retain classification capabilities when learning domain invariant features.
In the experiment, our architecture demonstrates better transfer ability in three transfer tasks of Office dataset. The accuracy of our proposed architecture in three transfer tasks are all about 0.3~0.5 higher than CAN.
關鍵字(中) ★ 遷移學習
★ 對抗式遷移學習
★ 類別不變性
★ 領域自適應
關鍵字(英) ★ Transfer Learning
★ Adversarial-based Transfer Learning
★ Class-invariant
★ Domain Adaption
論文目次 中文摘要(i)
Abstract(ii)
圖目錄(iii)
表目錄(v)
章節目錄 (vi)
緒論(P.1)
1.1 研究目的與背景(P.1)
1.2 研究方法(P.2)
1.3章節概要(P.3)
第二章、深度學習(P.4)
2.1 類神經網路(P.4)
2.2 深層神經網路(P.9)
2.3 深度卷積神經網路(P.11)
2.4 經典深度卷積網路架構(P.14)
第三章、遷移學習(P.19)
3.1遷移學習簡介(P.19)
3.2 非監督式領域自適應(P.21)
3.3深度神經網路的可遷移性(P.22)
3.4 基於映射式的深度遷移學習(Mapping-based Deep Transfer learning)(P.26)
3.5 基於對抗式深度遷移學習(Adversarial-based Deep Transfer learning)(P.32)
第四章、提出架構(P.37)
4.1 Collaborative and Adversarial Network for Unsupervised domain adaption (CAN)(P.37)
4.2 類別不變性密集對抗式遷移學習網路(P.40)
第五章、實驗結果(P.43)
5.1 實驗設置(P.43)
5.2 實驗結果(P.47)
5.3實驗分析(P.51)
第六章、結論與未來研究方向(P.54)
第七章、文獻(P.55)
參考文獻 [ 1 ] Y. Ganin and V. S. Lempitsky. “Unsupervised domain adaptation by backpropagation.” In International Conference on Machine Learning, 2015.
[ 2 ] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell. “Deep domain confusion: Maximizing for domain invariance.” arXiv preprint arXiv:1412.3474, 2014.
[ 3 ] M. Long, H. Zhu, J. Wang, and M. I. Jordan. “Unsupervised domain adaptation with residual transfer networks.” In Neural Information Processing Systems, 2016.
[ 4 ] Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. “Collaborative and adversarial network for unsupervised domain adaptation.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages. 3801–3809, 2018.
[ 5 ] Z. Pei, Z. Cao, M. Long, and J. Wang, “Multi-adversarial domain adaptation,” in Proc. 32nd AAAI Conf. Artif. Intell., New Orleans, LA, USA, 2018.
[ 6 ] F. Rosenblatt, “The perceptron: A probabilistic model for information storage and organization in the brain,” Psych. Rev., vol. 65, pages. 386–408, 1958.
[ 7 ] D. Rumelhart, G. Hinton, and R. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pages. 533–536, 1986.
[ 8 ] T. Tieleman, “Training Restricted Boltzmann Machines Using Approximations to the Likelihood Gradient,” Proc. Int’l Conf. Machine Learning, pages. 1064-1071, 2008.
[ 9 ] G. Hinton, S. Osindero, Y. Teh,“A Fast Learning Algorithm for Deep Belief Nets”Neural computation, Vol. 18, No. 7, pages. 1527-1554, 2006.
[ 10 ] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner,“Gradient-based learning applied todocument recognition,”Proceedings of the IEEE, vol. 86, no. 11, pages. 2278-2324, Nov 1998.
[ 11 ] A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Neural Information Processing Systems, 2012.
[ 12 ] Deng, J., et al. “ImageNet: A large-scale hierarchical image database” . IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[ 13 ] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. “Going deeper with convolutions”, In Computer Vision and Pattern Recognition, 2015.
[ 14 ] S. Ioffe and C. Szegedy. “Batch normalization: Accelerating deep network training by reducing internal covariate shift”, In International Conference on Machine Learning, 2015.
[ 15 ] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna.” Rethinking the Inception Architecture for Computer Vision”, ArXiv e-prints, December 2015.
[ 16 ] C. Szegedy, S. Ioffe, and V. Vanhoucke. “Inception-v4, inception-resnet and the impact of residual connections on learning”. In International Conference on Learning Representations Workshop, 2016.
[ 17 ] K. He, X. Zhang, S. Ren, and J. Sun. “Deep residual learning for image recognition”, In Computer Vision and Pattern Recognition, 2016
[ 18 ] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, ‘‘A survey on deep transfer learning,’’ in Proc. ICANN, Rhodes, Greece, 2018, pages. 270–279.
[ 19 ] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. “How transferable are features in deep neural networks?” In Advances in Neural Information Processing Systems, pages. 3320–3328, 2014.
[ 20 ] Transfer Learning - Machine Learning′s Next Frontier .http://ruder.io/transfer-learning/
[ 21 ] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. “Generative adversarial nets”, In Neural Information Processing Systems, pages. 2672–2680, 2014.
[ 22 ] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell. “Deep domain confusion: Maximizing for domain invariance”. CoRR, abs/1412.3474, 2014.
[ 23 ] M. Long, Y. Cao, J. Wang, and M. I. Jordan.” Learning transferable features with deep adaptation networks”, In Proceedings of the 32nd International Conference on Machine Learning, pages. 97–105, 2015.
[ 24 ] M. Long, J. Wang, and M. I. Jordan.” Unsupervised domain adaptation with residual transfer networks”, CoRR, abs/1602.04433, 2016.
[ 25 ] A. Krizhevsky, I. Sutskever, and G. Hinton. “Imagenet classification with deep convolutional neural networks”, In Neural Information Processing Systems, 2012.
[ 26 ] C. Zimmermann and T. Brox. “Learning to estimate 3d hand pose from single rgb images”. In International Conference on Computer Vision , 2017.
[ 27 ] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. “Attention is all you need”. In Neural Information Processing Systems, 2017.
[ 28 ] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size”. arXiv preprint arXiv:1602.07360, 2016
[ 29 ] O’Reilly, R. C. “Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm”, Neural Comput. 8, 895–938 (1996).
[ 30 ] H. Shimodaira. “Improving predictive inference under covariate shift by weighting the log-likelihood function”, Journal of statistical planning and inference, Vol. 90, No. 2, pages. 227–244, 2000.
[ 31 ] D. Dai and L. Van Gool. “Dark model adaptation: Semantic image segmentation from daytime to nighttime”, arXiv preprint arXiv:1810.02575, 2018
[ 32 ] Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, and Jan Kautz. “Joint discriminative and generative learning for person re-identification”, Computer Vision and Pattern Recognition, 2019.
[ 33 ] S. Woo, J. Park, J.-Y. Lee, and I. So Kweon. Cbam.” Convolutional block attention module”, In Proceedings of the European Conference on Computer Vision, pages 3–19, 2018
[ 34 ] X. Wang, R. Girshick, A. Gupta, and K. He. “Non-local neural networks”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7794– 7803, 2018.
[ 35 ] J. Hu, L. Shen, and G. Sun.”Squeeze-and-excitation networks”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[ 36 ] K. Saenko, B. Kulis, M. Fritz, and T. Darrell. “Adapting visual category models to new domains”, In European Conference on Computer Vision, pages 213– 226, 2010.
[ 37 ] He, Kaiming, and Jian Sun. “Convolutional neural networks at constrained time cost”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
[ 38 ] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, et al. “Tacotron: Towards end-to-end speech synthesis”, In Interspeech, 2017.
[ 39 ] A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. Smola. “A kernel two-sample test”, Journal of Machine Learning Research,Vol. 13, pages 723–773, March 2012.
[ 40 ] Y. Grandvalet and Y. Bengio. “Semi-supervised learning by entropy minimization”, In Neural Information Processing Systems, 2004.
[ 41 ] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. Decaf. ” A deep convolutional activation feature for generic visual recognition”, In International Conference on Machine Learning, 2014.
指導教授 王家慶(Jia-Ching Wang) 審核日期 2019-8-7
推文 facebook   plurk   twitter   funp   google   live   udn   HD   myshare   reddit   netvibes   friend   youpush   delicious   baidu   
網路書籤 Google bookmarks   del.icio.us   hemidemi   myshare   

若有論文相關問題,請聯絡國立中央大學圖書館推廣服務組 TEL:(03)422-7151轉57407,或E-mail聯絡  - 隱私權政策聲明