上年度計畫中，我們針對所提出的先導性HEVC編碼技術(半色調畫面內預測編碼和雙向絕對誤差和畫面間預測編碼)，探討研究利用支持向量機(SVM)應用於深度決策和模式決策中，以降低其計算複雜度。實驗結果顯示利用SVM技術在畫面內預測編碼可以減少22%的編碼計算時間，但平均只增加0.09% 的碼率(BDBR)。另外，在畫面間預測編碼部分，實驗結果也顯示SVM可以減少30%的編碼計算時間，但也只增加不到0.1% 的碼率。目前所提出的SVM機制，不但已經實現在HEVC參考軟體中，這些研究成果也發表在如GCCE..等重要的國際會議上。由於針對畫面內及畫面間預測編碼中深度和模式分類所開發出來的SVM方法有非常高的分類正確率(平均98%正確率) ，因此平均碼率只增加0.09% 而已。近幾年以卷積神經網路 (CNN)為基礎的深度學習技術已經廣泛地應用在HEVC編碼技術上，本計畫擬延續上年度計畫，我們將利用上年度已發展出來具高正確率分類的SVM技術與CNN 技術相結合並應用在HEVC編碼技術上。首先以發展出來具高正確率分類的SVM為基礎將影像分類成影像特性相近的各資料子集合，接著以卷積神經網路技術分別應用於各影像特性相近的資料子集合，如此更能訓練出更精確的模型，此結合SVM/CNN技術在編碼效率或者降低其計算複雜度方面皆可比單一CNN模型有更好的性能表現。第一年我們將探討研究此結合SVM/CNN新型架構在編碼效率之性能表現並與傳統單一CNN模型技術的性能比較。第二年我們將探討研究此結合SVM/CNN新型架構在降低編碼計算複雜度之性能表現並與傳統單純使用SVM 技術或CNN模型技術的性能比較。第三年我們將整合此新型SVM/CNN技術同時能增進改善編碼效率並降低其計算複雜度架構，將針對各量化參數(QP)找出最佳或次佳SVM/CNN模型，並實現在HEVC參考軟體中，並與原始HEVC編碼技術之編碼性能和計算複雜度作比較，探討其優劣。 ;In the last project we have been investigating using support vector machine (SVM) techniques for High Efficiency Video Coding (HEVC) to reduce its computational complexity, in special for previously proposed halftoned HEVC intra prediction and SABPD-based inter prediction. Experimental results demonstrate that average 22% of total encoding time can be reduced for HEVC intra prediction, but with only slightly increase in bit rate (only with average 0.09% bit rate increment). The results also show that 30% of computation time can be saved for HEVC inter prediction, and the bit rate increment is less than 0.1%, compared to the original HEVC. The proposed QP-optimized SVM algorithms have been realized in HEVC reference software, and parts of the results have been presented in IEEE international conferences such as GCCE etc. The developed QP-optimized SVM algorithm has very good coding performance, with less than 0.1% bit rate increment, compared to the original HEVC. This is due to that the SVM algorithm has very high classification accuracy (98% accuracy on average) to classify CU or PU modes. In recent years the convolutional neural network (CNN)-based deep learning technique has been widely used in HEVC to improve the coding performance or reduce the computational complexity. In this three year's project we will incorporate the developed SVM method into the CNN and will study the performance of the novel combined SVM/CNN architecture. Based upon the developed SVM algorithm we first classify a video sequence (coded tree blocks, CTUs) into some subgroups, and the CTUs within each subgroup has very high similar characteristics. Each subgroup is processed by its own CNN model and as a result more accurate and better CNN models can be obtained when compared with a single CNN model that is processed for all possible CTUs that has varieties of characteristics. In the first year, we will investigate the novel SVM/CNN architecture for HEVC to improve the coding performance. The coding performance will be studied and compared with that using CNN only. In the second year we will focus on reducing the computational complexity of HEVC with the new SVM/CNN architecture. The computation efficiency will be investigated and compared with those using SVM or CNN independently. In the last year, we will integrate all investigated algorithms together and investigate both coding efficiency and computation time, and compare with those using SVM or CNN only. We will also develop and implement new QP-optimized SVM/CNN architecture into HEVC reference software.
