本論文將提出一個基於生成對抗網路的演算法， 能夠重新生成一個人的各項細節至特定的姿態。 本研究的演算法包含(1)使用Pix2pix網路來將圖像從骨架圖片轉至對應UV座標圖片， (2)將人物的輪廓、UV座標圖片、以及原始圖片當作輸入，使用基於StyleGAN的網路來生成人物的圖像至指定姿態。 而根據本論文的實驗，本研究在使用骨架生成UV圖片的SSIM有0.932， 而在姿態與風格轉換上的SSIM有0.7524， 因此來證明本論文提供之演算法有一定程度之可用性。;In the past, pose re-rendering relied on skilled visual effects artists and time-consuming post-production. Traditional methods such as building 3D camera arrays to capture a human′s pose and build human keypoints to fit the animation model. Nowadays people use learning-based tools to generate images such as GAN(Generative Adversarial Network)s or other neural network frameworks. In order to capture human appearance, these methods tend to use skeleton, mesh, body part segmentation or dense UV coordinates to capture fine appearance details.
In this paper, we present a framework that could re-render a person from a single source image to a specific pose. Our framework includes (1) using Pix2pix network to generate UV coordinates image from a keypoint skeleton image. (2) Take human foreground mask, UV coordinate image and original images as input, use StyleGAN network to translate a person from source to target image.
According to the results of the experiments, the results of our skeleton keypoints to the UV coordinate model shows 0.932 on SSIM. And the results of our pose rerendering model shows 0.7524 on SSIM. Therefore, our framework has a certain degree of usability.
