基於內容分析之多運算子畫面尺寸調整與品質衡量機制

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：26

、訪客IP：3.133.120.226

姓名

韋岱延(Dai-Yan Wei) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於內容分析之多運算子畫面尺寸調整與品質衡量機制
(Content-Based Multi-Operator Retargeting and Its Quality Evaluation)

相關論文

★ 基於QT之跨平台無線心率分析系統實現	★ 網路電話之額外訊息傳輸機制
★ 針對與運動比賽精彩畫面相關串場效果之偵測	★ 植基於向量量化之視訊/影像內容驗證技術
★ 植基於串場效果偵測與內容分析之棒球比賽精華擷取系統	★ 以視覺特徵擷取為基礎之影像視訊內容認證技術
★ 使用動態背景補償以偵測與追蹤移動監控畫面之前景物	★ 應用於H.264/AVC視訊內容認證之適應式數位浮水印
★ 棒球比賽精華片段擷取分類系統	★ 利用H.264/AVC特徵之多攝影機即時追蹤系統
★ 利用隱式型態模式之高速公路前車偵測機制	★ 基於時間域與空間域特徵擷取之影片複製偵測機制
★ 結合數位浮水印與興趣區域位元率控制之車行視訊編碼	★ 應用於數位智權管理之H.264/AVC視訊加解密暨數位浮水印機制
★ 基於文字與主播偵測之新聞視訊分析系統	★ 植基於數位浮水印之H.264/AVC視訊內容驗證機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文研究提出基於畫面內容之多運算子影像尺寸調整機制，希望在顯示畫面於不同輸出設備時仍能保持畫質，本研究亦提出適用於此應用的畫質衡量模型，合理評估原始影像與修改後影像的差異。首先我們改良多運算子畫面調整機制SCAN，它包含了圖縫裁減(Seam carving)、邊緣裁切(Cropping)、增加圖縫(Add seams)與畫面縮放(Normalization)。本研究主要改善邊緣裁切步驟，透過前景物偵測將影像分類，根據類別及畫面中的物體以不同的視覺顯著圖決定適當裁切位置。此外，我們加入人臉與建築物偵測，避免出現於畫面邊緣的人臉可能遭受不當裁切，並判斷建築物是否為畫面重要內容。實驗結果顯示所提出的改良式多運算子畫面調整機制在各式影像中能有效維持內容完整。在畫質衡量模型中，我們利用SIFT Flow比較原始影像及濃縮影像的內容差異，考量可能出現的幾何扭曲及線段扭曲，根據畫面顯著物及語意相關程度，以類神經網路迴歸分析找出平均意見分數(MOS)對每種屬性的依據，進而得到更貼近於人眼主觀感受的評估。實驗結果顯示，與其他評估方法相較，我們所提出的模型更貼近於MOS的結果。

摘要(英)

This research proposes a content-based multi-operator image retargeting scheme, enabling the retargeted images to preserve its content after adaptation in various displays. Besides, a quality evaluation model is also proposed to compare original images and retargeted images. The proposed multi-operator retargeting scheme is termed “SCAN” as it contains Seam caving, Cropping, Adding seams and Normalization (scaling). This research mainly concentrates on improving the step of content-based cropping in SCAN. We classify images into two categories via foreground detection and adopt different types of visual saliency to determine appropriate cropping limits. The face detection is also introduced to protect face areas appearing at the edges of an image from being removed. A building detection mechanism is employed to determine whether a building in an image is significant or not. The experimental shows that the improved multi-operator retargeting scheme can effectively preserve the content and objects’ shape when dealing with various images. In the proposed quality evaluation model, we make use of SIFT Flow to compare the contents of original and retargeted images and identify possible geometric distortion and line distortion. We further consider salient objects and image semantics in the evaluation process. With these attributes, we utilize the neural network regression model to determine the weights of every feature in order to fit the Mean Opinion Score (MOS). The results show that the proposed model is closer to MOS than other evaluation methods.

關鍵字(中)

★ 多運算子畫面調整機制
★ 前景物偵測
★ 濃縮影像品質衡量
★ SIFT Flow
★ 線段扭曲
★ 幾何扭曲
★ 迴歸分析

關鍵字(英)

★ Multi-Operators
★ Foreground Detection
★ Retarget
★ Quality Evaluation
★ SIFT Flow
★ Line Distortion
★ Geometric Distortion
★ Regression Analysis

論文目次

論文摘要 i
Abstract ii
Content iii
List of Figures vi
List of Tables ix
Chapter 1. Introduction 1
1.1 Motivation 1
1.2 Contribution 5
1.3 Thesis Organization 6
Chapter 2. Related Work 7
2.1 Image Retargeting 7
2.1.1 Common Content-Based Retargeting Methods 7
2.1.2 SCAN 9
2.2 Performance Evaluating of Retargeting 13
2.2.1 Subjective Evaluation 13
2.2.2 Objective Evaluation 13
Chapter 3. Improved SCAN 16
3.1 Overview 16
3.2 Object Detection 17
3.2.1 Foreground Detection 18
3.2.2 Building Detection 19
3.2.3 Face Detection 19
3.3 Visual Saliency Map 20
3.3.1 Visual Saliency Feature 20
3.3.2 Deep Saliency 23
3.4 Foreground Extraction 25
3.5 Content-Based Image Retargeting Scheme 27
3.5.1 Improved Content-Based Cropping 27
3.5.2 Cropping Limits Refinement 30
Chapter 4. Quality Evaluation Scheme 33
4.1 Overview 33
4.2 Preprocessing 34
4.3 Line Distortion 35
4.4 Geometric Distortion 37
4.5 Distortion Analysis 38
4.6 Image Semantics Analysis 39
4.6.1 Saliency 39
4.6.2 Semantic Segmentation 40
4.7 Regression 42
Chapter 5. Experiment Results 44
5.1 Retargeting Mechanism 44
5.1.1 Image Classification 44
5.1.2 Results of Content-Based Cropping 46
5.1.3 Comparison with Ours and Other Retargeting Schemes 49
5.2 Quality Evaluation Model 60
5.2.1 Comparison with Subjective and Objective Scores 60
5.2.2 Scores of Other Retargeting Mechanism and SCAN 64
Chapter 6. Conclusion and Future Work 69
Reference 71

參考文獻

[1] D. Vaquero, M. Turka, K. Pullib, M. Ticob, and N. Gelfandb, "A survey of image retargeting techniques," Proceedings of SPIE the International Society for Optical Engineering, vol. 7798, p. 779814, 2010.
[2] Ce Liu, Yuen, J. and Torralba, A. (2011). “SIFT Flow: Dense Correspondence across Scenes and Its Applications.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), pp.978-994.
[3] F. Stentiford, "Attention based auto image cropping," The 5th International Conference on Computer Vision Systems, Bielefeld, 2007.
[4] I. S. Amrutha, S. S. Shylaja, S. Natarajan, and K. N. Murthy, "A smart automatic thumbnail cropping based on attention driven regions of interest extraction," Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, ACM, pp. 957-962, 2009.
[5] P. Cheatle, "Automatic image cropping for republishing," IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, pp. 75400O-75400O-9, 2010.
[6] L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, 1998.
[7] R. Gal, O. Sorkine, and D. Cohen-Or, "Feature-aware texturing," Proceedings of the 17th Eurographics conference on Rendering Techniques, Eurographics Association, June 2006.
[8] Y. S. Wang, C. L. Tai, O. Sorkine, and T. Y. Lee, "Optimized scale-and-stretch for image resizing," ACM Transactions on Graphics (TOG), vol. 27, no. 5, 2008.
[9] S. Avidan, and A. Shamir, "Seam carving for content-aware image resizing," ACM Transactions on graphics (TOG), vol. 26, no. 3, Aug 2007.
[10] M. Rubinstein, A. Shamir, and S. Avidan, "Improved seam carving for video retargeting," ACM Transactions on Graphics (TOG), vol. 27, no. 3, p. 16, 2008.
[11] M. Grundmann, V. Kwatra, M. Han, and I. Essa, "Discontinuous seam-carving for video retargeting," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, June 2010.
[12] D. Domingues, A. Alahi, and P. Vandergheynst, "Stream carving: an adaptive seam carving algorithm," 17th IEEE International Conference on Image Processing (ICIP), pp. 901-904, 2010.
[13] S. Hua, G. Chen, H. Wei, and Q. Jiang, "Similarity measure for image resizing using SIFT feature," EURASIP Journal on Image and Video Processing, pp. 1-11, Jan 2012
[14] M. Rubinstein, A. Shamir, and S. Avidan, "Multi-operator media retargeting," ACM Transactions on Graphics (TOG), vol. 28, no. 3, 2009.
[15] W. Dong, N. Zhou, J. C. Paul, and X. Zhang, "Optimized image resizing using seam carving and scaling," ACM Transactions on Graphics (TOG), vol. 28, no. 5, p.125, 2009.
[16] Y.C., Chou, P. C., Su, “Toward More Efficient Multi-Operator Media Retargeting for Digital Images and Videos”, National Central University, 2016.
[17] S. Montabone, and A. Soto. “Human detection using a mobile platform and novel features derived from a visual saliency mechanism,” Image and Vision Computing, vol. 28, no. 3, pp391-402
[18] Ma, L., Lin, W., Deng, C. and Ngan, K. (2012). “Image Retargeting Quality Assessment: A Study of Subjective Scores and Objective Metrics.” IEEE Journal of Selected Topics in Signal Processing, 6(6), pp.626-639.
[19] Yabin Zhang, Weisi Lin, Qiaohong L, Wentao Cheng , and Xinfeng Zhang. “Multiple-Level Feature-Based Measure for Retargeted Image Quality” IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 1, pp 451-463, JANUARY 2018
[20] Yabin Zhang, Weisi Lin, Yuming Fang, Leida Li, “ASPECT RATIO SIMILARITY (ARS) FOR IMAGE RETARGETING QUALITY ASSESSMENT.”
[21] Yabin Zhang, Yuming Fang, Weisi Lin, Xinfeng Zhang and Leida Li, “Backward Registration-Based Aspect Ratio Similarity for Image Retargeting Quality Assessment,” IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 25, NO. 9, pp. 4286-4297, SEPTEMBER 2016
[22] Hsu, C., Lin, C., Fang, Y. and Lin, W. (2014). “Objective Quality Assessment for Image Retargeting Based on Perceptual Geometric Distortion and Information Loss.” IEEE Journal of Selected Topics in Signal Processing, 8(3), pp.377-389.
[23] S. Ren, K. He, R. Girshick, and J. Sun, ‘‘Faster R-CNN: Towards realtime object detection with region proposal networks,’’ IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017
[24] PASCAL VOC 2012 [Online] Available: http://host.robots.ox.ac.uk/pascal/VOC/
[25] Leeds Butterfly Dataset. Josiah Wang, Katja Markert, and Mark Everingham. Learning Models for Object Recognition from Natural Language Descriptions. In Proceedings of the 20th British Machine Vision Conference (BMVC2009) [Online] Avaliable: http://www.josiahwang.com/dataset/leedsbutterfly/
[26] 17 Category Flower Dataset. Maria-Elena Nilsback and Andrew Zisserman [Online] Available: http://www.robots.ox.ac.uk/~vgg/data/flowers/17/
[27] WIDER FACE: A Face Detection Benchmark [Online] Available: http://shuoyang1213.me/WIDERFACE/
[28] Pingping Zhang Dong Wang Huchuan Lu Hongyu Wang Xiang Ruan Dalian University of Technology, China Tiwaki Co.Ltd, “Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection”
[29] MSRA10k salient Object Database. Available: https://mmcheng.net/msra10k
[30] Hengshuang Zhao, Jianping Shi Xiaojuan Qi Xiaogang Wang, Jiaya Jia. “Pyramid Scene Parsing Network,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 2881-2890, 2017.
[31] RetargetMe Benchmark [Online]. Available: http:// http://people.csail. mit.edu/mrub/retargetme/index.html
[32] NTHU Retargeting Image Dataset (NRID) [Online]. Available: http://www.ee.nthu.edu.tw/cwlin/Retargeting_Quality/NRID.html
[33] Zhibo Chen, Jianxin Lin, Ning Liao, and Chang Wen Chen. “Full Reference Quality Assessment for Image Retargeting Based on Natural Scene Statistics Modeling and Bi-Directional Saliency Similarity,” IEEE Transaction on Image Processing vol. 26, no. 11, pp 5138-5148, 2017

指導教授

蘇柏齊(Po-Chyi Su)

審核日期

2019-8-13

推文