人臉動畫化 : 臉部關鍵點辨識與感知色彩距 離之特徵加權循環生成對抗網路;Face Animation: Feature Weighted CycleGAN With Facial Landmark Recognition and Perceptual Color Distance

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/92534

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/92534

Title:	人臉動畫化 : 臉部關鍵點辨識與感知色彩距離之特徵加權循環生成對抗網路;Face Animation: Feature Weighted CycleGAN With Facial Landmark Recognition and Perceptual Color Distance
Authors:	羅士倫;Lo, Shih-Lun
Contributors:	資訊工程學系
Keywords:	生成對抗網路;動畫人臉風格轉換;Generative Adversarial Network;Anime Face Style Transfer
Date:	2023-07-20
Issue Date:	2023-10-04 16:04:17 (UTC+8)
Publisher:	國立中央大學
Abstract:	隨著近年日本動畫的熱潮，透過虛擬動畫角色與觀眾互動的新興行業 Vtuber 有非常大的商業潛力，然而，角色的建模過程十分複雜，本篇論文將以生成對抗網路進行人臉風格轉換解決此問題。而人臉影像風格轉換在電腦視覺領域中是一項困難的任務，動畫人臉與真人人臉在結構上具有明顯的差異，如何將兩者間進行轉換，同時保留相似特徵非常具有挑戰性。在架構上，我們以 U-GAT-IT 模型為基礎，修改其中標準化方法以得到更多特徵資訊，並且提出 Facial Landmark Loss 計算生成影像與真人五官位置誤差，來幫助模型學到更精確的五官位置，而對於 U-GAT-IT 本身色彩偏差問題，我們則使用可微分之 CIEDE2000 色差公式作為損失函數來得到更加符合人眼色彩感知的影像。在評估上，由於現階段沒有合理的指標足以評估動畫角色的真實程度，我們提出 Fréchet Anime Inception Distance 計算在高維空間中生成動畫影像與真實動畫影像在分佈上的距離，藉此來了解生成動畫影像品質的好壞。最後，根據實驗結果與使用者表單回饋，我們所提出的方法在多項指標上，均有較好的表現。;With the boom of Japanese animation in recent years, the emerging industry of Vtuber, which interacts with the audience through virtual animation characters, has great commercial potential. However, the process of creating character model is complicated. With the significant difference between human face and anime face, image style conversion is a difficult task in the field of computer vision. In this paper, we will solve the problem of face style conversion by generative adversarial network. Our model is based on U-GAT-IT and modify the normalization function to obtain more feature information. To make the face feature position of anime face similar to human face, we propose Facial Landmark Loss to calculate the error between the generated image and real human face image. Because of the obvious color deviation of images of U-GAT-IT, we introduced Perceptual Color Loss into loss function. Since there is no reasonable metrics to evaluate the realism of the animated images, we propose Fréchet Anime Inception Distance to calculate the distance between the distribution of the generated animated images and the real animated images in high- dimensional space, so as to understand the quality of the generated animated images. According to the experimental results and user feedback, our proposed method has a better performance in many metrics.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	39	View/Open

社群 sharing

Loading...