發展更具效率之多運算子影像視訊畫面尺寸調整機制

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：50

、訪客IP：18.191.103.46

姓名

周永杰(Yung Chieh Chou) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

發展更具效率之多運算子影像視訊畫面尺寸調整機制
(Toward More Efficient Multi-Operator Media Retargeting for Digital Images and Videos)

相關論文

★ 基於QT之跨平台無線心率分析系統實現	★ 網路電話之額外訊息傳輸機制
★ 針對與運動比賽精彩畫面相關串場效果之偵測	★ 植基於向量量化之視訊/影像內容驗證技術
★ 植基於串場效果偵測與內容分析之棒球比賽精華擷取系統	★ 以視覺特徵擷取為基礎之影像視訊內容認證技術
★ 使用動態背景補償以偵測與追蹤移動監控畫面之前景物	★ 應用於H.264/AVC視訊內容認證之適應式數位浮水印
★ 棒球比賽精華片段擷取分類系統	★ 利用H.264/AVC特徵之多攝影機即時追蹤系統
★ 利用隱式型態模式之高速公路前車偵測機制	★ 基於時間域與空間域特徵擷取之影片複製偵測機制
★ 結合數位浮水印與興趣區域位元率控制之車行視訊編碼	★ 應用於數位智權管理之H.264/AVC視訊加解密暨數位浮水印機制
★ 基於文字與主播偵測之新聞視訊分析系統	★ 植基於數位浮水印之H.264/AVC視訊內容驗證機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本研究提出多運算子影像與視訊尺寸調整(retargeting)演算法，目的在於有效率地調整影像畫面至目標解析度，並將演算法延伸應用於視訊。對於數位影像，我們適當地施予基於內容之邊緣裁切(content-based cropping)和縮放(scaling)，首先計算影像中的視覺顯著特徵(visual saliency feature)，並將影像透過SLIC(Simple Linear Iterative Clustering)演算法切割成較大的超級像素(superpixel)，擷取畫面中的前景物作為畫面切割的依據，接著逐一比較視覺特徵圖進行邊緣裁切與等比例縮放。若時間允許，圖縫裁減(seam carving)可被使用讓畫面更接近目標長寬比。圖縫裁減主要計算畫面梯度，採用動態規劃刪除最小能量圖縫並進行圖縫的局部更新，最後定義突出點以限制圖縫數量並決定裁減停止點。對於某些適合的影像，我們亦可增加圖縫來降低畫面直接縮放程度。由實驗結果顯示，我們確實有效率地維持影像主體，演算法也達到較高的實用性。另外，我們將影像處理延伸至視訊資料，考量視訊壓縮域動態資料計算，透過H.264/AVC視訊壓縮編碼時所產生的運動向量(motion vector)和運動補償資訊(motion compensation)判斷鏡頭種類，若為非固定式場景，我們使用邊緣裁切以及縮放的方式處理畫面；若為固定場景，則可使用圖縫裁減機制。為了防止運動中的前景物在裁切過程中被移除而造成失真，我們將壓縮域中的位移向量製作運動特徵圖(motion feature map)，結合視覺特徵圖協助圖縫裁減和邊緣裁切。實驗結果顯示我們的方法可以廣泛處理不同種類的鏡頭，在畫面前景物形狀的維持以及背景保留上，亦優於其他視訊畫面調整演算法。

摘要(英)

This research presents a multi-operator image retargeting scheme, which can be further expanded to video retargeting. The objective is to effectively and efficiently adjust the image or video frame to the targeted resolution. Given an image or frame, the content-based cropping and scaling will be applied. The visual saliency map is calculated and the superpixels are formed via Simple Linear Iterative Clustering (SLIC) to serve as the reference to extract the visually significant foreground objects. Next, the degree of cropping and scaling will be determined by the saliency map. Seam carving can also be employed to make the resolution closer to the target if the efficiency is not an important issue. Seam caving checks the one-directional gradients and uses dynamic programming to remove the saliency with minimal significance. Local update helps to reduce the computational burden. Saliency points are identified and helps to decide when to stop the seam carving process. For certain images, inserting seams is also useful to decrease the the degree of scaling. Experimental results show that the proposed method does maintain the significant objects of the image and is also more feasible.

For video retargeting, the data in compressed video stream, including the motion vectors and motion compensation, are used to classify the types of shots. If the shot belongs to a fixed scene, seam carving can be applied. Otherwise, only cropping and scaling are used. To avoid removing the foreground objects, the motion feature map is formed, combined with the visual saliency map, to achieve seam carving and cropping. The experimental results shows that the proposed scheme can deal a variety of shots and outperform existing algorithms.

關鍵字(中)

★ 多種運算子
★ 邊緣裁切
★ 圖縫裁減
★ 視覺顯著特徵
★ 運動向量
★ 運動特徵圖

關鍵字(英)

★ Multi-Operator
★ Content-based Cropping
★ Seam Carving
★ Visual Saliency Map
★ H.264 Motion Vector
★ Motion Feature Map

論文目次

論文摘要 i
Abstract ii
誌謝 iv
目錄 v
附圖目錄 viii
附表目錄 xi
第一章緒論 1
1.1 研究背景與動機 1
1.2 研究貢獻 3
1.3 論文架構 4
第二章相關研究 5
2.1 影像畫面調整機制 5
2.1.1邊緣裁切方法 5
2.1.2影像變形方法 6
2.1.3圖縫裁減方法 6
2.1.4多種運算子方法 7
2.2 視訊畫面調整機制 8
2.2.1基於內容之縮放和區塊變形 9
2.2.2加強版圖縫裁減 9
第三章多運算子影像尺寸調整機制 11
3.1 系統流程 11
3.2 視覺顯著特徵 12
3.3 超級像素切割 15
3.4 基於內容之邊緣裁切 19
3.4.1前景物擷取 19
3.4.2連通數分析 20
3.4.3裁切邊緣分析 22
3.4.4特殊狀況 : 模糊背景切割 24
3.5 圖縫裁減 27
3.5.1 圖縫連接的方法 27
3.5.2 空間關聯度衡量 31
3.5.3 長斜直線偵測 34
3.5.4 顯著特徵點以及圖縫裁減停止點分析 36
3.5.5 圖縫添加 39
3.6 影像調整原則分析 41
第四章多運算子之視訊調整機制 42
4.1 系統流程 42
4.2 H.264/AVC壓縮域資料分析 43
4.2.1 壓縮域運動預測和運動補償 43
4.2.2 視覺重要度分析 46
4.3 基於內容之視訊畫面調整 49
4.3.1 視訊鏡頭分類 49
4.3.2 固定鏡頭調整機制 51
4.3.3 非固定鏡頭調整機制 53
第五章實驗結果 56
5.1 影像畫面調整機制 56
5.1.1 視覺顯著圖(VSF) 56
5.1.2 基於內容之影像裁切實驗結果 58
5.1.3 圖縫裁減實驗結果 61
5.1.4 實驗結果比較 63
5.1.5 影像調整演算法品質分析和執行效率比較 69
5.2 視訊畫面調整機制 74
5.2.1 視訊分類實驗結果 74
5.2.2 固定鏡頭畫面調整實驗結果 75
5.2.3 非固定鏡頭畫面調整實驗結果 77
5.2.4 視訊鏡頭連續畫面比較 80
第六章結論與未來方向 83
6.1 結論 83
6.2 未來方向 84
參考文獻 85

參考文獻

[1] D. Vaquero, M. Turka, K. Pullib, M. Ticob, and N. Gelfandb, "A survey of image retargeting techniques," Proceedings of SPIE the International Society for Optical Engineering, vol. 7798, p. 779814, 2010.
[2] M. Rubinstein, D. Gutierrez, and O. Sorkine, "A comparative study of image retargeting," ACM Transactions on Graphics (TOG), vol. 29, no. 6, p. 160, 2010.
[3] S. Montabone, and A. Soto., "Human detection using a mobile platform and novel features derived from a visual saliency mechanism," Image and Vision Computing, vol. 28, no. 3, pp. 391-402, 2010.
[4] L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, 1998.
[5] M. Zhang, and L. Zhang, "Auto cropping for digital photographs," IEEE International Conference on Multimedia and Expo, p. 4-pp, 2005.
[6] F. Stentiford, "Attention based auto image cropping," The 5th International Conference on Computer Vision Systems, Bielefeld, 2007.
[7] I. S. Amrutha, S. S. Shylaja, S. Natarajan, and K. N. Murthy, "A smart automatic thumbnail cropping based on attention driven regions of interest extraction," Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, ACM, pp. 957-962, 2009.
[8] P. Cheatle, "Automatic image cropping for republishing," IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, pp. 75400O-75400O-9, 2010.
[9] A. Santella , M. Agrawala, D. DeCarlo, D. Salesin, and M. Cohen, "Gaze-based interaction for semi-automatic photo cropping," Proceedings of the SIGCHI conference on Human Factors in computing systems. pp. 771-780, ACM, 2006.
[10] H. Liu, X. Xie, W. Y. Ma, and H. J. Zhang,"Automatic browsing of large pictures on mobile devices,"Proceedings of the eleventh ACM international conference on Multimedia. pp. 148-155, 2003.
[11] B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobjs, "Automatic thumbnail cropping and its effectiveness," Proceedings of the 16th annual ACM symposium on User interface software and technology, 2003.
[12] G. Ciocca, C. Cusano, F. Gasparini, and R.Schettini, "Self-adaptive image cropping for small displays," IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1622-1627, 2007.
[13] S. Avidan, and A. Shamir, "Seam carving for content-aware image resizing," ACM Transactions on graphics (TOG), vol. 26, no. 3, Aug 2007.
[14] M. Rubinstein, A. Shamir, and S. Avidan, "Improved seam carving for video retargeting," ACM Transactions on Graphics (TOG), vol. 27, no. 3, p. 16, 2008.
[15] M. Grundmann, V. Kwatra, M. Han, and I. Essa, "Discontinuous seam-carving for video retargeting," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, June 2010.
[16] D. Domingues, A. Alahi, and P. Vandergheynst, "Stream carving: an adaptive seam carving algorithm," 17th IEEE International Conference on Image Processing (ICIP), pp. 901-904, 2010.
[17] R. Gal, O. Sorkine, and D. Cohen-Or, "Feature-aware texturing," Proceedings of the 17th Eurographics conference on Rendering Techniques. Eurographics Association, June 2006.
[18] L. Wolf, M. Guttmann, and D. Cohen-Or, "Non-homogeneous content-driven video-retargeting," IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, 2007.
[19] Y. S. Wang, C. L. Tai, O. Sorkine, and T. Y. Lee, "Optimized scale-and-stretch for image resizing," ACM Transactions on Graphics (TOG), vol. 27, no. 5, 2008.
[20] S. Hua, G. Chen, H. Wei , and Q. Jiang, "Similarity measure for image resizing using SIFT feature," EURASIP Journal on Image and Video Processing, pp. 1-11, Jan 2012.
[21] M. Rubinstein, A. Shamir, and S. Avidan, "Multi-operator media retargeting," ACM Transactions on Graphics (TOG), vol. 28, no. 3, 2009.
[22] W. Dong, N. Zhou, J. C. Paul, and X. Zhang, "Optimized image resizing using seam carving and scaling," ACM Transactions on Graphics (TOG), vol. 28, no. 5, p.125, 2009.
[23] Y. S. Wang, H. C. Lin, O. Sorkine, and T. Y. Lee, "Motion-based video retargeting with optimized crop-and-warp," ACM Transactions on Graphics (TOG), vol. 29, no. 4, p.90, 2010.
[24] S. Luo, J. Zhang, Q. Zhang, and X. Yuan, "Multi-operator image retargeting with automatic integration of direct and indirect seam carving," Image and Vision Computing, 2012.
[25] W. M. Dong, G. B. Bao, X. P. Zhang, and J. C. Paul, "Fast multi-operator image resizing and evaluation," Journal of Computer Science and Technology, vol. 27, no. 1, pp. 121-134, 2012.
[26] Huan Du, Zhi Liu, Zhiguo Yan, and Yayun Jiang, "Video retargeting based on stretchability aware block scaling," IEEE conference, The Third Research Institute of Ministry of Public Security, 2014.
[27] Lingqiang Ran, Na Lv, and Xiangxu Meng, "Image retargeting based on spring analogy, " International Conference on Progress in Informatics and Computing, 2014.
[28] Yanwen Guo, Feng Liu, Jian Shi,Zhi-Hua Zhou, and Michael Gleicher, "Image retargeting using mesh parametrization," IEEE Transactions on Multimedia, vol. 11, no. 5, August 2009.
[29] Zhiquan Ren, Botao Wang, Yuchen Zhang, Hongkai Xiong, "Visual preserving video retargeting with deformable shape consistency," IEEE conference, Department of Electronic Engineering Shanghai Jiao Tong University, 2013.
[30] BoYan, Kairan Sun, Liu Liu, "Matching-area-based seam carving for video retargeting, " IEEE Transactions on circuits and systems for video technology, vol. 23, no. 2, February 2013.
[31] Wei-Lun Chao, Hsiao-Hang Su, Shao-Yi Chien, Winston Hsu3, and Jian-Jiun Ding, "Coarse-to-fine temporal optimization for video retargeting based on seam carving," International Conference on Multimedia and Expo (ICME), pp. 1-6, 2011.
[32] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk, "SLIC Superpixels," EPFL Technical Report, no. 149300, June 2010.
[33] Jiangyang Zhang, Shangwen Li, C.-C. Jay Kuo, "Compressed-domain video retargeting," IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 797-809, February 2014.
[34] R. Wang, H.-J. Zhang, and Y.-Q. Zhang, "A confidence measure based moving object extraction system built for compressed domain," Circuits and Systems, Proceedings. ISCAS 2000 Geneva, The 2000 IEEE International Symposium, vol. 5, pp. 21–24, May 2000.
[35] Chih-Chung Hsu, Chia-Wen Lin, Yuming Fang, Weisi Lin, "Objective quality assessment for image retargeting based on perceptual distortion and information loss," IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 3, pp. 377-389, 2014.
[36] RetargetMe Benchmark [Online]. Available: http:// http://people.csail. mit.edu/mrub/retargetme/index.html
[37] Renjie Chen, Daniel Freedman, Zachi Karni, Craig Gotsman, Ligang Liu, "Content-aware image resizing by quadratic programming," Computer Vision and Pattern Recognition Workshops (CVPRW), July 2010.
[38] Z. Karni, D. Freedman, C. Gotsman, "Energy-based image deformation," Eurographics Symposium on Geometry Processing, vol. 28, no.5, pp. 1257-1268, July 2009.
[39] Yuanyuan Ding, Jing Xiao, Jingyi Yu, "Importance filtering for image retargeting," Computer Vision and Pattern Recognition (CVPR), pp. 89-96, June 2011.
[40] NTHU Retargeting Image Dataset (NRID) [Online]. Available: http://www.ee.nthu.edu.tw/cwlin/Retargeting_Quality/NRID.html
[41] Xiph.org Video Test Media [Online]. Available: https://media.xiph.org/video/derf/

指導教授

蘇柏齊(Po Chyi Su)

審核日期

2016-8-29

推文