隨著電腦運算能力提升，與機器學習模型的發展，影像相關的模型有了長 足的進展。在生成對抗網路被提出後，各種影像生成的應用應運而生，動態影 片生成便是其中之一。從影像序列生成動態影片，可以協助系統分析場景中物 件的未來動態，並成為系統決策的依據，如：自動駕駛系統可以藉此判斷潛在 危險，搶先駕駛人察覺之前做出應對。也可以用於生成創作素材，由單張影像 生成動態影片。目前以循環輸入的方法，生成動態影片的相關研究，主要分為 兩種：第一種，模型直接生成影像；第二種，模型生成合成資訊，再與輸入影 像合成。直接生成影像的方法，模型同時要學習生成真實影像與動態變化；而 生成合成資訊的方法，專注於學習合成相鄰影像之間的動態，因此通常可以生 成較清晰的影像，本研究研讀過往研究決定採用此方法。然而，無論何種方 法，在連續循環輸入模型生成一定時間長度後，影像都會變得模糊。為了使生 成影像維持清晰，本研究提出長短時間步長模型，先由長時間步長模型使用光 流法 (Optical Flow)，跳躍設定的步長生成影像，再以短時間步長模型使用差異 法 (Difference) 生成中間的影像。在海浪資料集與行車紀錄器資料集，進行單 張與多張影像輸入的實驗。兩資料集所得到的評估分數，都顯示本研究提出的 模型，能夠減緩連續循環生成影像產生的模糊區塊，使影片在生成較長時間 後，仍保有影像結構與合理的動態。;Generating motion video from image sequence helps system to analyze the motion of objects in the scene and make decision. For example, self-driving system can determine whether there is a potential danger and react before driver being aware of it. The application can also be used to generate video materials from single image input. In recent research, there are mainly two ways to generate motion video by recurrently take generated image as input. Firstly, directly generate image with model. Secondly, synthesize the generated warping information and the last image of input sequence. For the first method, the network has to model not only the distribution of training data but also the object motion simultaneously. For the second method, the network focus on learning how to generate the warping information between adjacent frames. We decide to adopt the second method since it generally generates sharper images than the first method. However, both recurrent generating methods mentioned above will eventually produce blurry result after generating several frames. In order to keep generated image sharp, we propose long and short time step generative adversarial network. Firstly, the long time step network generate the long time step frames skipping a number of frame number with optical flow method. Secondly, generate the frames between two long time step frames with difference method. We conduct single and multi-image sequence experiments on sea wave dataset and car camera dataset. The result shows that our model is able to generate images with less blurry artifact, stable scene and reasonable motion after generating several frames.
