近年虛擬實境(virtual reality, VR)興起，360度視訊(360 degree video)提供使用者身臨其境的體驗，而由於360度視訊須由球體域投影至二維平面(sphere-to-plane projection)，以致不同區域有不均勻的幾何形變(geometric deformation)，因此，幾乎現有基於傳統2D影像而設計的視訊分析與處理方案，於此類投影影像皆無法達到最佳效能。雖然現今360度視訊壓縮標準相關研究顯示，cubemap (CMP) 投影影像的編碼rate-distortion (R-D)效能優於equirectangular projection (ERP)影像，但因球體域上均勻取樣的各點，非均勻地投影至CMP的各面(face)的不同區域，其相關編碼方案仍需改進，其中，目前幾乎無任何已發表文獻探討編碼CMP投影影像的位元率控制(rate control)。另一方面，視覺追蹤為許多電腦視覺應用必要的前端處理，但現有的視覺追蹤方案，卻僅極少數基於360度視訊之CMP投影影像而設計。由於近年電腦視覺領域研究顯示深度學習(deep learning)可有效提升其準確率，因此，本一年期計畫之研究目標，為對360度視訊的CMP投影影像處理與分析的兩個主題提出設計: (1)基於深度學習的視訊編碼(video coding)之位元率控制(rate control)。(2) 基於深度學習的視覺追蹤(visual tracking)。本計畫研究成果預期將有效提升編碼CMP投影影像的位元率控制之準確率，與CMP投影影像的視覺追蹤準確率，進而提升360度視訊相關應用之品質。 ;With the rise of virtual reality, 360 degree videos provide users immersive experiences. However, sphere-to-plane projections of 360 degree videos lead to non-uniform geometric deformations in various regions of 2-D projected images. Accordingly, 2-D projections of 360 degree videos degrade performance of existing video analysis and processing schemes that designed for conventional 2-D images. Although the state-of-the-art in 360-degree video coding indicates that the R-D performance of the cubemap (CMP) projection outperforms that of the equirectangular projection (ERP), the uniform samples in the sphere domain are still projected non-uniformly onto different regions of each face of the CMP projection. Thus, the existing CMP projection based encoder can be further improved while there is few rate control scheme designed for CMP encoding. On the other hand, visual tracking is essential to many applications of computer vision. However, there are few trackers designed for the CMP projection based 360 degree videos. Since research results indicate that deep learning can significantly improve performance of applications of computer vision, this project focuses on two topics of the CMP projection of 360 degree video processing and analysis: (1) Deep learning based rate control of video coding. (2) Deep learning based visual tracking. For the CMP projection of 360 degree videos, this project expects to increase accuracy of rate control of video coding and accuracy of visual tracking. Accordingly, quality of the 360 degree video related applications can be significantly improved.
