不規則圖像轉換及基於分散誤差之影片縫切割技術

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：94

、訪客IP：3.149.249.154

姓名

張閎翰(Hon-Hang Chang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

不規則圖像轉換及基於分散誤差之影片縫切割技術
(Irregular Image Transformation and Balanced Error Distribution for Video Seam Carving)

相關論文

★ 基於edX線上討論板社交關係之分組機制	★ 利用Kinect建置3D視覺化之Facebook互動系統
★ 利用 Kinect建置智慧型教室之評量系統	★ 基於行動裝置應用之智慧型都會區路徑規劃機制
★ 基於分析關鍵動量相關性之動態紋理轉換	★ 基於保護影像中直線結構的細縫裁減系統
★ 建基於開放式網路社群學習環境之社群推薦機制	★ 英語作為外語的互動式情境學習環境之系統設計
★ 基於膚色保存之情感色彩轉換機制	★ 一個用於虛擬鍵盤之手勢識別框架
★ 分數冪次型灰色生成預測模型誤差分析暨電腦工具箱之研發	★ 使用慣性傳感器構建即時人體骨架動作
★ 基於多台攝影機即時三維建模	★ 基於互補度與社群網路分析於基因演算法之分組機制
★ 即時手部追蹤之虛擬樂器演奏系統	★ 基於類神經網路之即時虛擬樂器演奏系統

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

近年來，越來越多的圖像和視頻重新定位技術被提出來，特別是基於使用 seam carving、warping 或結合兩者之技術，成功地在多媒體領域上豐富我們的日常生活。雖然這些技術之已發展多年，但這些影像重新定位技術仍僅適用在於將不同尺寸之矩形與矩形間作圖像轉換，其不能輕易地重新將矩形圖片轉換成另一個圓形、多邊形及其他形狀。因此，本論文著提出一適用於不規則影像至轉換之圖像轉換系統命名為 CMAIR，此變形轉換系統能夠將矩形圖像重新定位為不同形狀的圖像，並能突顯主要興趣的顯著物件。在我們的圖像轉換系統中，我們另外提出了一種獨特的不規則插值方法來產生四個可能的目標圖像，並可經由我們們所提出之影像評估機制，再考慮圖像顯著性下來確定最佳候選圖像作為最終輸出。經由實驗結果發現，我們的實驗結果不僅可以將源圖像放置到不同的目標形狀的蒙版中，而且還可以盡可能保留和突出顯著的物件，幫助我們能夠瀏覽更清晰之影像。本文所提出的算法其重點在於，能夠在視圖片內容中考量不規則形狀之影像內容物之結構，並將影像其內容物合理的放置於所給定之框架中。

在影片的重新定位部分，本研究另外提出一個影片雕刻技術，目的在於加強相連影像於時間域中保持其影像內容物之連續性，並避免影片內容物之顯著主體被破壞，加強視覺無違和之視頻產出。同時本研究特別針對其複雜影像內容物的視頻 (包含存在透視效果之消失縣、存在大量線性結構之影像內容）來證明此演算法的實用性。

由本論文所提供之視訊結果中可以明顯的發現，該圖片或影像在影像縮小之過程中能有效地保持及維護其圖像或影格內容之線性結構，並在影片中的各影格維持其視覺之連續性；由圖像之結果可以證明本研究所提出的不規則縮放技術，讓矩形圖片之影像內容能夠有效的向內填入於不規則之外框，以不規則之影像外型來呈現。為了驗證本研究方法之可行性，http://video.minelab.tw/
DETS/EDSeamCarving/ 及 http://video.minelab.tw/DETS/VRDSF/ 介紹本論文視訊與圖像在尺寸調整後其影像內容物之比較來驗證本論文方法的貢獻。

摘要(英)

In recent years, more and more image retargeting techniques have been proposed to facilitate our daily life, in particular those based on the use of seam carving, warping or the combination of them. Although these techniques are rather sophisticated, they only work for a specific pattern during retargeting. For example, these techniques can only retarget the source picture into the same shape of a square, but cannot be easily reshaped into a circular, a polygon or other shapes. This paper focuses on creating a graphics editing system, named CMAIR (Content and Mask-Aware Image Retargeting), for image retargeting, which can retarget the source images into diff erent shapes of image to highlight the salient objects of primary interest. CMAIR eff ectively supports removal of unimportant pixels, and frames as many surrounding objects inside the provided mask as possible. In this paper, we propose a unique irregular interpolation method to produce four possible target images, and an evaluation mechanism to decide the best candidate image as the final output with the consideration of image saliency. The results show that not only the source image can be placed into different targeted shapes of mask, but also the salient objects are retained and highlighted as much as possible. As a result, the salient objects become more clear in our
eyes.

Besides, a video retargeting method named BED is proposed in this study, and the proposed algorithm in this thesis focuses on maintaining the structure of straight lines and irregular shape of objects without deforming complex image contents, which may be altered in traditional seam carving of complex image or video. In addition, the proposed mechanism maintains visual continuity such that the resulting videowill not look shaky due to sudden changes in the background. Practical applicability of the proposed method was tested using both regular videos and special videos which contain vanishing lines (i.e., perspective eff ects).

Experimental results demonstrate that the proposed CMAIR can convert the rectangular image to another irregular shape of image, and all the salient objects of the targeted images are preserved. In our video results using BED, the BED approach not only resizes the video with the retention of important information, but also maintains the structural properties of objects in varios kinds of videos.

The demonstration website at http://video.minelab.tw/DETS/
EDSeamCarving/ and http://video.minelab.tw/DETS/VRDSF/ provides comparison results that illustrate the contribution of our method.

關鍵字(中)

★ 重新定位技術
★ 縫切割
★ 能量圖
★ 顯著圖

關鍵字(英)

★ Retargeting
★ Seam Carving
★ Energy Map
★ Saliency Map

論文目次

Pages
Chinese abstract . . . . . . . . . . . . . . . . . . . . . . . . . i
English abstract . . . . . . . . . . . . . . . . . . . . . . . . . iii
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . v
Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . viii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Symbol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . 1
1-1 Image Retargeting . . . . . . . . . . . . . . . . . 1
1-2 Video Retargeting . . . . . . . . . . . . . . . . . 3
Chapter 2 Related Works . . . . . . . . . . . . . . . . . . . 9
2-1 Image Retargeting . . . . . . . . . . . . . . . . . 9
2-2 Video Retargeting . . . . . . . . . . . . . . . . . 12
Chapter 3 Content and Mask-Aware Image Retargeting . . 16
3-1 Energy Creating System . . . . . . . . . . . . . 17
3-1-1 Mask Map (MM) Definition . . . . . . . . . . . 17
3-1-2 Energy Definition . . . . . . . . . . . . . . . . . 18
3-2 Image retargeting system . . . . . . . . . . . . . 21
3-2-1 Removal and duplication of seams . . . . . . . . 21
3-2-2 Definition of Pair-seam . . . . . . . . . . . . . . 22
3-2-3 Pair-Seam Carving . . . . . . . . . . . . . . . . 23
3-2-4 Termination condition and algorithm . . . . . . 24
3-3 Irregular Interpolation System . . . . . . . . . . 24
3-3-1 Content-and-mask-aware evaluation mechanism . 27
Chapter 4 Balanced Error Distribution for Video Seam Carving . . . . . . . . . . . . . . . . . . . . . . . . . 29
4-1 Seam Carving . . . . . . . . . . . . . . . . . . . 29
4-2 Spatial Coherence and Error Distribution . . . . 31
4-3 Temporal Coherence . . . . . . . . . . . . . . . 36
4-4 Image Results . . . . . . . . . . . . . . . . . . . 38
4-4-1 Body maintenance . . . . . . . . . . . . . . . . 38
4-4-2 Maintenance of Linear/Geometry Structures . . 41
4-5 Video Test Results . . . . . . . . . . . . . . . . 44
4-6 Limitation of Seam Carving Based Retargeting
Methods . . . . . . . . . . . . . . . . . . . . . . 53
Chapter 5 Conclusion and Future Works . . . . . . . . . . 55
Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

參考文獻

[1] Lior Wolf, Moshe Guttmann, and Daniel Cohen-Or. Non-
homogeneous content-driven video-retargeting. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on,pages 1–6. IEEE, 2007.
[2] Yu-Shuen Wang, Hongbo Fu, Olga Sorkine, Tong-Yee Lee, and Hans-Peter Seidel. Motion-aware temporal coherence for video resizing. ACM Transactions on Graphics (TOG), 28(5):127, 2009.
[3] Johannes Kiess, Stephan Kopf, Benjamin Guthier, and Wolfgang Effelsberg. Seam carving with improved edge preservation. In IS&T/SPIE Electronic Imaging, pages 75420G–75420G. International Society for Optics and Photonics, 2010.
[4] Matthias Grundmann, Vivek Kwatra, Mei Han, and Irfan Essa. Discontinuous seam-carving for video retargeting. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 569–576. IEEE, 2010.
[5] Michael Rubinstein, Ariel Shamir, and Shai Avidan. Improved seam carving for video retargeting. In ACM transactions on graphics (TOG), volume 27, page 16. ACM, 2008.
[6] Yu-Shuen Wang, Jen-Hung Hsiao, Olga Sorkine, and Tong-Yee Lee. Scalable and coherent video resizing with per-frame optimization. In ACM Transactions on Graphics (TOG), volume 30, page 88. ACM, 2011.
[7] Shih-Syun Lin, Chao-Hung Lin, I-Cheng Yeh, Shu-Huai Chang, Chih-Kuo Yeh, and Tong-Yee Lee. Content-aware video retargeting using object-preserving warping. IEEE transactions on visualization and computer graphics, 19(10):1677–1686, 2013.
[8] Tzu-Chieh Yen, Chia-Ming Tsai, and Chia-Wen Lin. Maintaining temporal coherence in video retargeting using mosaic-guided scaling. IEEE Transactions on Image Processing, 20(8):2339–2351, 2011.
[9] Yu-Shuen Wang, Hui-Chih Lin, Olga Sorkine, and Tong-Yee Lee. Motion-based video retargeting with optimized crop-and-warp. In ACM Transactions on Graphics (TOG), volume 29, page 90. ACM, 2010.
[10] Shaoyu Qi, Yu-Tseh Jason Chi, Adrian M Peter, and Jeff rey Ho. Casair: Content and shape-aware image retargeting and its applications. IEEE Transactions on Image Processing, 25(5):2222–2232, 2016.
[11] Anthony Santella, Maneesh Agrawala, Doug DeCarlo, David Salesin, and Michael Cohen. Gaze-based interaction for semiautomatic photo cropping. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 771–780. ACM, 2006.
[12] Bongwon Suh, Haibin Ling, Benjamin B Bederson, and David W Jacobs. Automatic thumbnail cropping and its effectiveness. In Proceedings of the 16th annual ACM symposium on User interface software and technology, pages 95–104. ACM, 2003.
[13] Hao Liu, Xing Xie, Wei-Ying Ma, and Hong-Jiang Zhang. Automatic browsing of large pictures on mobile devices. In Proceedings of the eleventh ACM international conference on Multimedia, pages 148–155. ACM, 2003.
[14] Stephan Kopf, Benjamin Guthier, Hendrik Lemelson, and Wolfgang Effelsberg. Adaptation of web pages and images for mobile applications. In IS&T/SPIE Electronic Imaging, pages 72560C–72560C. International Society for Optics and Photonics, 2009.
[15] Feng Liu and Michael Gleicher. Video retargeting: automating pan and scan. In Proceedings of the 14th ACM international conference on Multimedia, pages 241–250. ACM, 2006.
[16] Feng Liu and Michael Gleicher. Automatic image retargeting with fisheye-view warping. In Proceedings of the 18th annual ACM symposium on User interface software and technology, pages 153–162. ACM, 2005.
[17] Sebastiano Battiato, Giovanni Maria Farinella, Nicolo Grippaldi, and Giovanni Puglisi. Content-based image resizing on mobile devices. In VISAPP (2), pages 87–90, 2012.
[18] Chenjun Tao, Jiaya Jia, and Hanqiu Sun. Active window oriented dynamic video retargeting. In Proceedings of the Workshop on Dynamical Vision, ICCV, 2007.
[19] Sunghyun Cho, Hanul Choi, Yasuyuki Matsushita, and Seungyong Lee. Image retargeting using importance diff usion. In Image Processing (ICIP), 2009 16th IEEE International Conference on, pages 977–980. IEEE, 2009.
[20] Li-Qun Chen, Xing Xie, Xin Fan, Wei-Ying Ma, Hong-Jiang Zhang, and He-Qin Zhou. A visual attention model for adapting images on small displays. Multimedia systems, 9(4):353–364, 2003.
[21] Xin Fan, Xing Xie, He-Qin Zhou, and Wei-Ying Ma. Looking into video frames on small displays. In Proceedings of the eleventh ACM international conference on Multimedia, pages 247–250. ACM, 2003.
[22] Jianping Xiao, Xuecheng Zou, Zhenglin Liu, and Xu Guo. A novel adaptive interpolation algorithm for image resizing. International Journal of Innovative Computing, Information and Control, 3(6), 2007.
[23] Yunfeng Zhang, Shanshan Gao, Caiming Zhang, and Jing Chi. Application of a bivariate rational interpolation in image zooming. Int. J. Innov. Comput. Inform. Cont, 5(11), 2009.
[24] Ping-Hsien Lin, Hung-Kuo Chu, Tong-Yee Lee, et al. Smooth shape interpolation for 2d polygons. International Journal of Innovative Computing Information and Control, 4(9):2405–2417, 2008.
[25] Yan Tian, Caifang Zhang, Fuyuan Peng, and Sheng Zheng. An iterative hybrid method for image interpolation. Advances in Intelligent Computing, pages 10–19, 2005.
[26] Shai Avidan and Ariel Shamir. Seam carving for content-aware image resizing. In ACM Transactions on graphics (TOG), volume 26, page 10. ACM, 2007.
[27] Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee. Optimized scale-and-stretch for image resizing. In ACM Transactions on Graphics (TOG), volume 27, page 118. ACM, 2008.
[28] Bo Yan, Kairan Sun, and Liu Liu. Matching-area-based seam carving for video retargeting. IEEE Transactions on Circuits and Systems for Video Technology, 23(2):302–310, 2013.
[29] Laurent Itti, Christof Koch, and Ernst Niebur. A model of saliencybased visual attention for rapid scene analysis. IEEE Transactions on pattern analysis and machine intelligence, 20(11):1254–1259, 1998.
[30] Yuming Fang, Zhenzhong Chen, Weisi Lin, and Chia-Wen Lin. Saliency detection in the compressed domain for adaptive image retargeting. IEEE Transactions on Image Processing, 21(9):3888–3901, 2012.
[31] Michael Rubinstein, Ariel Shamir, and Shai Avidan. Multi-operator media retargeting. In ACM Transactions on Graphics (TOG), volume 28, page 23. ACM, 2009.
[32] Shih-Syun Lin, I-Cheng Yeh, Chao-Hung Lin, and Tong-Yee Lee. Patch-based image warping for content-aware retargeting. IEEE transactions on multimedia, 15(2):359–368, 2013.
[33] Sebastiano Battiato, Giovanni Maria Farinella, Giovanni Puglisi, and Daniele Ravi. Saliency-based selection of gradient vector flow paths for content aware image resizing. IEEE Transactions on Image Processing, 23(5):2081–2095, 2014.
[34] Gallea Roberto, Edoardo Ardizzone, and Roberto Pirrone. Real-
time content-aware image resizing using reduced linear model. In
Image Processing (ICIP), 2010 17th IEEE International Conference
on, pages 2813–2816. IEEE, 2010.
[35] Zachi Karni, Daniel Freedman, and Craig Gotsman. Energy-based image deformation. In Computer Graphics Forum, volume 28, pages 1257–1268. Wiley Online Library, 2009.
[36] Philipp Krähenbühl, Manuel Lang, Alexander Hornung, and Markus Gross. A system for retargeting of streaming video. In ACM Transactions on Graphics (TOG), volume 28, page 126. ACM, 2009.
[37] Bing Li, Ling-Yu Duan, Jinqiao Wang, Rongrong Ji, Chia-Wen Lin, and Wen Gao. Spatiotemporal grid fl ow for video retargeting. IEEE Transactions on Image Processing, 23(4):1615–1628, 2014.
[38] Xiaodi Hou, Jonathan Harel, and Christof Koch. Image signature: Highlighting sparse salient regions. IEEE transactions on pattern analysis and machine intelligence, 34(1):194–201, 2012.
[39] Ali Borji, Ming-Ming Cheng, Huaizu Jiang, and Jia Li. Salient object detection: A benchmark. IEEE Transactions on Image Processing, 24(12):5706–5722, 2015.
[40] Yuanyuan Ding, Jing Xiao, and Jingyi Yu. Importance fi ltering for image retargeting. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 89–96. IEEE, 2011.
[41] Radhakrishna Achanta and Sabine Süsstrunk. Saliency detection for content-aware image resizing. In Image Processing (ICIP), 2009 16th IEEE International Conference on, pages 1005–1008. IEEE,
2009.

指導教授

施國琛(Timothy K. Shih)

審核日期

2017-7-31

推文