| 摘要: | 影像的分辨率越高，可以檢索到的信息越詳細。 因此，分辨率小的影像會丟失很多重要的細節和資訊。 儘管此資訊可用於各種工作，例如分類、檢測、追蹤或分割。 我們已經看到使用深度學習技術（尤其是生成對抗網絡 (GAN)）來提高超分辨率影像質量的好的成果。 然而，值得注意的是，在輸入和輸出具有明顯不同尺度的情況下進行超解析度處理將增加一定的難度。 在本文中，我們提出了一種多步驟的超解析度方法，通過逐漸調整影像的尺度來實現最佳效果。影片超解析度（VSR）與單幀影像超解析度（SISR）存在不同的問題，因此需要採用不同的方法。大多數現有研究對於各幀之間的關系沒有的完全理解，但在VSR中，這一點至關重要，需要更多的關註。於此，時間特徵可以提供很多好處，在VSR中，它可以保持影片的品質和運動連續性，以及影片的一致性。通過這種方式，我們能夠避免影像中不一致性的錯誤隨時間累積的問題，而此問題是由於時間損失函數導致的。由我們的超分辨率方法的最終結果可以看出，它產生的超分辨率影片在使用此方法時，既保持影片質量又保持其運動連續性。此外，我們的結果在各種資料集中超越了SOTA方法。我們使用的其中一個資料集是煙花資料集。煙花有一個獨特的特點，在VSR中，它可以展現出清晰的動態，而在分割問題上，由於它不是固定單一的物體，而是會分離，具有動態的形狀和顏色，因此非常具有挑戰性。然而，還有其他需要解決的障礙，即缺乏具有煙花分割標簽的資料。因此，我們採用無監督學習的方法來自動進行分割。
關鍵詞：影片超分辨率，無監督分割，多跳，長期損失，影像處理，生成對抗網絡。
;The higher the resolution of an image, the more detailed information can be retrieved. Therefore, an image with a small resolution will lose a lot of important details and information. Even though this information can be used for various jobs such as classification, detection, tracking or segmentation. We have seen promising results from using deep learning techniques, especially Generative Adversarial Networks (GANs), to enhance the quality of super-resolution images. However, it is important to note that performing super resolutions with input and output that have a significantly different scale will add a certain amount of difficulty. Throughout this paper we propose the use of a super resolution that has multiple steps, which scales the image gradually in order to achieve maximum results. Video super resolution (VSR) has different problems with single image super resolution (SISR) so it requires a different approach. Most of the existing studies do not have a complete understanding of the relationship between frames, but in VSR this is crucial and needs more attention. There are numerous benefits that can be derived from the temporal feature, in VSR it can maintain video quality and motion continuity, as well as video consistency. In this way, we are able to avoid the inconsistent failures in the image which accumulate over time as a result of the temporal loss functions. The final result of our super resolution method can be seen in the fact that it produces a super-resolution video that maintains both the video′s quality as well as its motion continuity when using our proposed method. Moreover, our result is surpassing the state-of-the-art method in various datasets. One of the datasets that we use is Fireworks dataset. The fireworks have a unique characteristic, on VSR it can show the clear motion and on segmentation problem, it is very challenging due to the fireworks is not a solid object, also has a dynamic shape and color. However, there are other obstacles that need to be resolved, namely the unavailability of data that has segmentation labels for fireworks. Therefore, we take an unsupervised learning approach to be able to do segmentation automatically.
Keywords: Video super resolution, Unsupervised Segmentation, Multi-hop, long-term loss, Image Processing, Generative adversarial network. |
