基於弱監督式學習可變形模型之三維人臉重建

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：31

、訪客IP：3.144.113.197

姓名

李品萱(Pin-Hsuan Li) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

基於弱監督式學習可變形模型之三維人臉重建
(3D Face Reconstruction based on Weakly-Supervised Learning Morphable Face Model)

相關論文

★ 具有元學習分類權重轉移網路生成遮罩於少樣本圖像分割技術

★ 具有注意力機制之隱式表示於影像重建三維人體模型

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 ( 永不開放)

摘要(中)

在現今生活周遭，由於技術的成熟、儀器設備完善，許多產業的應用逐漸從二維平面拓展至三維空間，例如：影視娛樂產業、人臉識別、醫學美容等相關領域，以實現更加真實的環境，提供大眾絕佳的感官體驗。

傳統三維人臉重建算法無法深入學習到真實人臉特徵，並且太過依賴圖像的低層次資訊，而一般業界所使用的軟體建模法、儀器建模法是相當耗費人力與時間，以及需要花費大筆金額。因此，近幾年基於深度學習的三維人臉重建技術在品質和效率方面都展示出很好的成效，在質量跟時間方面達到平衡，重建出對應圖像之三維人臉模型。

深度神經網路的訓練通常需要大量的訓練數據，但目前含有真實三維人臉模型的數據非常少，因此本論文使用的方法為弱監督式學習，結合卷積神經網路模型以及三維可變形模型針對圖像進行深度的特徵提取及回歸相關係數並利用可變形模型重建以及改善其前處理和損失函數，利用圖像高、低層次的資訊，在沒有真實標籤的情況下，能有效地重建圖像人臉，對於角度變化豐富或是含有遮擋之圖像也能獲得很好的重建品質，後續將在兩個數據集中評估及分析其實驗結果，並且展示重建之三維人臉模型。

摘要(英)

Nowadays, due to the maturity of technology and the perfection of instruments and equipment, the application of many industries has gradually expanded from two-dimensional plane to three-dimensional space, such as video entertainment industry, face recognition, medical cosmetology and other related fields, in order to realize a more realistic environment and provide excellent sensory experience for the public.

Traditional face reconstruction algorithms are unable to learn real face features in depth and rely on low-level information of the image, while the software modeling method and instrument modeling method used by the industry are time-consuming and costly. Therefore, in recent years, 3D face reconstruction techniques based on deep learning have shown effective results in terms of quality and efficiency, balancing quality and time to reconstruct 3D face models corresponding to images.

Therefore, this paper uses a weakly-supervised approach that combines a Convolutional Neural Network and a 3D Morphable Face Model. The face can be effectively reconstructed without realistic labels by using the depth feature extraction and regress coefficients of the image, and improving the pre-processing and loss function. The reconstruction results will be evaluated and analyzed in two datasets, and the reconstruction results will be demonstrated.

關鍵字(中)

★ 深度學習
★ 三維人臉重建
★ 三維可變形模型

關鍵字(英)

★ Deep Learning
★ 3D Face Reconstruction
★ 3D Morphable Face Model

論文目次

摘要----------------------------i
Abstract------------------------ii
誌謝----------------------------iii
目錄----------------------------v
圖目錄--------------------------viii
表目錄--------------------------xi
第一章緒論----------------------1
1-1 研究背景--------------------1
1-2 研究動機與目的--------------2
1-3 論文架構--------------------3
第二章三維人臉重建技術相關介紹----4
2-1 三維重建技術概述------------5
2-2 三維人臉重建相關介紹---------6
2-2-1 軟體建模法----------------6
2-2-2 儀器掃描建模法------------8
2-2-3 基於三維可變形模型建模法---12
2-3 總結三維人臉重建技術--------19
第三章深度學習相關介紹----------20
3-1 類神經網路-----------------21
3-1-1 類神經網路學習機制--------21
3-1-2 類神經網路發展-----------24
3-2 卷積神經網路---------------29
3-2-1 卷積層-------------------30
3-2-2 批歸一化及激活函數--------32
3-2-3 池化層-------------------35
3-2-4 全連接層-----------------36
第四章提出之架構----------------38
4-1 系統架構-------------------38
4-2 訓練模型階段----------------39
4-2-1 圖像數據集及前處理--------40
4-2-2 神經網路模型架構----------48
4-2-3 損失函數-----------------59
4-3 測試模型階段----------------63
第五章實驗結果與分析討論---------64
5-1 實驗環境與數據集介紹--------64
5-2 實驗結果比較與討論----------66
第六章結論與未來展望------------78
參考文獻------------------------80

參考文獻

[1] Yu-Wen Chang, “A Review of 3D Structured Light Measurement Methods and Applications,” 2021.
[2] V. Blanz and T. Vetter, “A morphable model for the synthesis of 3D faces,” In Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pages 187–194,1999.
[3] P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, “A 3D face model for pose and illumination invariant face recognition,” In IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS), pages 296–301,2009.
[4] T. Gerig et al., “Morphable Face Models - An Open Framework,” 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018.
[5] Cao Chen, Yanlin Weng, Shun Zhou, Yiying Tong, Kun Zhou, “FaceWarehouse: a 3D Facial Expression Database for Visual Computing,” IEEE Transactions on Visualization and Computer Graphics, 20(3): 413-425, 2014.
[6] Bagdanov, Andrew D. and Masi, Iacopo and Del Bimbo, Alberto, “The Florence 2D/3D Hybrid Face Dataset,” Proc. of ACM Multimedia Int.’l Workshop on Multimedia access to 3D Human Objects (MA3HO’11), 2011.
[7] Z. H. Zhou, “A brief introduction to weakly supervised learning,” National Science Review, vol. 5, no. 1, pp. 44-53, 2017.
[8] Warren Sturgis McCulloch and Walter Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bulletin of Mathematical Biophysics, vol.5, no.4, pp.115-133, Dec. 1943.
[9] D. O. Hebb, “Organization of Behavior,” New York: Wiley & Sons.
[10] F. Rosenblatt, “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Cornell Aeronautical Laboratory, Psychological Review, v.65, no.6, pp.386-408.
[11] P. J. Werbos, “Beyond regression: new tools for prediction and analysis in the behavioral sciences,” Ph.D. thesis, Harvard University, 1974.
[12] K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biological cybernetics, vol. 36, no. 4, pp. 193-202, 1980.
[13] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-based learning applied to document recognition,” in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
[14] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” In Proceedings of The 32nd International Conference on Machine Learning, pages 448–456, 2015.
[15] Liu, Ziwei and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou, “Deep Learning Face Attributes in the Wild,” Proceedings of International Conference on Computer Vision (ICCV), Dec. 2015.
[16] T. Karras, S. Laine and T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 2019.
[17] Jian Zhao and Hui Zhang, “Thin-Plate Spline Motion Model for Image Animation,” 2022 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[18] Jiankang Deng, Jia Guo, Evangelos Ververas, Irene Kotsia, Stefanos Zafeiriou, “RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5203-5212, 2020.
[19] M. J. Jones and J. M. Rehg, “Statistical color models with application to skin detection,” International Journal of Computer Vision (IJCV), 46(1):81-96, 2002.
[20] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
[21] J. Deng, J. Guo, N. Xue and S. Zafeiriou, “ArcFace: Additive Angular Margin Loss for Deep Face Recognition,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4685-4694, 2019.
[22] Y. Deng, J. Yang, S. Xu, D. Chen, Y. Jia and X. Tong, “Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 285-295, 2019.
[23] K. Zhang, Z. Zhang, Z. Li and Y. Qiao, “Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks,” in IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, Oct. 2016.
[24] F. Schroff, D. Kalenichenko and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815-823, 2015.
[25] 2D以及3D妝容輸入圖片來源https://www.yuntuiweishang.com/knowledge/show-475.html
https://www.bilibili.com/s/video/BV1F64y1t7VT

指導教授

張寶基陳永芳(Pao-Chi Chang Yung-Fang Chen)

審核日期

2022-8-6

推文