樂器表演虛擬換衣系統：以吉他為例;Instruments perform Virtual try-on system: using Guitar as an Example

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/92478

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/92478

Title:	樂器表演虛擬換衣系統：以吉他為例;Instruments perform Virtual try-on system: using Guitar as an Example
Authors:	劉起華;Hua, Liu-Chi
Contributors:	資訊工程學系
Keywords:	虛擬換衣;人體分割;深度學習;Virtual try-on;Human Parsing;Deep learning
Date:	2023-07-11
Issue Date:	2023-10-04 16:02:41 (UTC+8)
Publisher:	國立中央大學
Abstract:	深度學習已應用在多個領域之中，在圖像領域上更是取代了多個傳統技術，虛擬穿衣(Virtual try-on)便是在該領域上的重要分支，2D應用上常用在商業成衣產業線上試衣，減少消費者到現場的成本，在3D應用中，則會生成一個3D的人體模型，在模型上進行試穿，輸出結果上比2D方法更加穩定，但須繁雜的預處理，如3D掃描等。在本文中，我們提出一個樂器表演換衣系統，可以讓使用者在改變在樂器表演影片中的外衣，可用於短影音娛樂與其他使用者分享，不需重新換裝。此系統使用深度學習的人體分割技術，以SCHP與DensePose作為主要人體分割模型，將人體與衣著各部位傳遞給穿衣模型。因人體分割無法表示被遮蔽的人體，使用OpenPose作為人體與手部骨架系統填補缺失的人體資訊。HR-VITON作為主要穿衣深度模型，利用人體分割與人體骨架的資訊，產生合理的穿衣結果。因HR-VITON為了提高模型泛化能力，會將部分影像挖空作為輸入，來讓結果在邊界的處理更加平滑，但這會影響樂器的還原能力，本文調整了影像挖空的演算法，讓樂器仍保良好的還原效果，讓使用者能在已有表演影像的前提下，做出更多不同的變化。;Deep learning has been applied in various domains and has superseded several traditional techniques in the field of image processing. Virtual try-on is an important sub-domain in this field. In 2D applications, Virtual try-on is often used in the online fitting of commercial garments, reducing the cost for consumers to visit physical locations. In 3D applications, a 3D human body model is generated and used for fitting, resulting in more stable outputs than 2D methods, but requiring complex preprocessing such as 3D scanning. In this paper, we propose an instrument performance virtual try-on system that allows users to change their clothing in instrument performance videos, suitable for sharing short video entertainment with other users without changing clothes. The system uses deep learning-based human body segmentation techniques, with SCHP and DensePose as the main body segmentation models, and transfers body and clothing parts to the Virtual try-on model. Since human body segmentation cannot represent occluded body parts, OpenPose is used as a body and hand skeleton system to supplement missing body information. HR-VITON serves as the main Virtual try-on model, using body segmentation and body skeleton information to generate reasonable fitting results. To improve the model′s generalization ability, HR-VITON introduces a hole digging technique for input images to achieve smoother boundary handling. However, this negatively affects the instrument restoration capability. In this study, we adjust the hole digging algorithm to maintain good instrument restoration effects, enabling users to create various changes based on existing performance videos.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	36	View/Open

社群 sharing

Loading...