利用區塊壓縮特徵一致性之編碼影像竄改和視訊偽造偵測

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：100

、訪客IP：3.149.247.95

姓名

張凱名(Kai-Ming Chang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

利用區塊壓縮特徵一致性之編碼影像竄改和視訊偽造偵測
(Evaluating Block Consistency by Compression Features for Forgery Detection of Encoded Images and Videos)

相關論文

★ 基於QT之跨平台無線心率分析系統實現	★ 網路電話之額外訊息傳輸機制
★ 針對與運動比賽精彩畫面相關串場效果之偵測	★ 植基於向量量化之視訊/影像內容驗證技術
★ 植基於串場效果偵測與內容分析之棒球比賽精華擷取系統	★ 以視覺特徵擷取為基礎之影像視訊內容認證技術
★ 使用動態背景補償以偵測與追蹤移動監控畫面之前景物	★ 應用於H.264/AVC視訊內容認證之適應式數位浮水印
★ 棒球比賽精華片段擷取分類系統	★ 利用H.264/AVC特徵之多攝影機即時追蹤系統
★ 利用隱式型態模式之高速公路前車偵測機制	★ 基於時間域與空間域特徵擷取之影片複製偵測機制
★ 結合數位浮水印與興趣區域位元率控制之車行視訊編碼	★ 應用於數位智權管理之H.264/AVC視訊加解密暨數位浮水印機制
★ 基於文字與主播偵測之新聞視訊分析系統	★ 植基於數位浮水印之H.264/AVC視訊內容驗證機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2026-8-15以後開放)

摘要(中)

由於影像編輯工具和基於深度學習的偽造生成應用的普及，人們可
以輕易地修改影像和視訊並將其散播至社交媒體網路。竄改的影像及偽
造的視訊不僅混淆視聽，對於個人名譽或身分隱私更可能造成無法挽回
的損害。近年多種影像竄改內容定位技術和相關深偽視訊偵測方法相繼
被提出，現有方法通常針對目標竄改手法畫面進行訓練而產生針對性的
偵測機制。然而偽造技術與時俱進，竄改內容的方法可能是未知的，且竄
改影像及視訊在網路上傳播時又可能經過壓縮編碼等處理致使竄改痕跡
消失，讓現有方法的偵測結果無法令人信服。
本研究提出使用區塊一致性的深度學習偽造偵測方法，針對壓縮影
像和編碼視訊中偽影減少問題，透過評估畫面中區塊內容的相似性來判
斷影像或視訊是否受到竄改。所提出的方法不針對各式竄改操作的資料
進行訓練，僅使用一般的影像資料訓練深度學習模型以實現相關的偵測
辨別，降低模型泛化能力不足的疑慮。我們透過卷積神經網路，設計能夠
分辨來源影像的通用特徵提取器，並利用孿生網路進行影像壓縮程度分
類。對於影像竄改偵測，我們使用熱力圖轉換和前景提取技術定位竄改區
域。而對於視訊偽造偵測，我們針對人臉周圍區域，透過比對前後幀的相
似程度來判斷該視訊的真實性。本研究方法在公開的資料集上進行驗證
和測試以證明其可行性，這些資料集包含各式竄改影像及深偽視訊模型
之輸出，代表不同類型的影像和視訊，藉此顯示此區塊壓縮特徵一致性方
法有助於影像竄改偵測與深偽視訊識別。

摘要(英)

With the growing accessibility of image editing tools and deep learning-based forgery applications, individuals can easily alter images and videos, disseminating them across social media networks. Such tampered images and forged videos not only create confusion but can also cause irreversible damage to personal reputation and privacy. In response, numerous detection methods for forged images and deepfake videos have been developed in recent years. These methods often rely on training with datasets containing specific tampering techniques to create targeted detection mechanisms. However, as forgery technologies advance, new and unknown tampering methods may emerge. Additionally, tampered images and videos may undergo compression or encoding during dissemination, which can obscure tampering traces, diminishing the effectiveness of current detection methods.
This study introduces a deep learning-based forgery detection method that utilizes block consistency to address the challenge of diminished tampering traces in compressed images and encoded videos. By evaluating the similarity of block content within the images or videos, this method determines whether tampering has occurred. Unlike existing approaches that train on the datasets with target tampering operations, our method uses general image data to train the deep learning model, thereby enhancing the model’s generalization capability. The proposed scheme was formed by first developing a feature extractor using convolutional neural networks to identify the source of the images and then employing a Siamese network to classify image compression levels. For image tampering detection, heatmap transformations and foreground extraction were used to pinpoint tampered areas. In deepfake video detection, we concentrated on facial regions, assessing the similarity between consecutive frames to verify the video’s authenticity. The effectiveness of this method was validated and tested using publicly available datasets, which include a range of tampered images and outputs from deepfake video models. The strong performance of the proposed block consistency method underscores its potential in enhancing image tampering detection and deepfake video identification.

關鍵字(中)

★ 深度學習
★ 影像竄改
★ 深度偽造
★ 孿生網路

關鍵字(英)

論文目次

論文摘要 I
Abstract II
致謝 IV
目錄 V
圖目錄 VII
表目錄 VIII
第一章、緒論 1
1.1. 研究背景和動機 1
1.2. 研究貢獻 2
1.3. 論文架構 3
第二章、相關研究 4
2.1. 數位影像鑑識 4
2.1.1. 傳統檢測方法 5
2.1.2. 噪聲殘差(Noise residuals) 5
2.1.3. 基於JPEG偽影之檢測方法 6
2.1.4. 監督式深度學習之檢測方法 7
2.2. 深度偽造偵測 ( Deepfakes Detection) 8
2.2.1. 影像竄改檢測 8
2.2.2. 視訊偽造檢測 9
第三章、提出方法 11
3.1. 系統概述 11
3.2. 特徵提取器 13
3.2.1. 噪聲殘差提取網路 13
3.2.2. 壓縮品質分類 17
3.3. 影像竄改偵測 19
3.3.1. 可視化與區域評比變因應對 19
3.3.2. 閥值限界選定與遮罩修正 21
3.4. Deepfake影片偽造檢測 24
第四章、實驗結果 30
4.1. 開發環境 30
4.2. 訓練資料 30
4.3. 竄改影像偵測成果 32
4.3.1. 偵測結果展示 32
4.3.2. 效能評估 34
4.4. 偽造視訊偵測成果 38
第五章、結論與未來展望 41
5.1. 結論 41
5.2. 未來展望 41
參考文獻 43

參考文獻

[1] S. Fazli, M. Moeini, "A robust image watermarking method based on DWT, DCT, and SVD using a new technique for correction of main geometric attacks," in Optik, vol. 127, no. 2, pp. 964-972, 2016.
[2] I. Yerushalmy, H. Hel-Or, "Digital image forgery detection based on lens and sensor aberration," in International journal of computer vision, vol. 92, pp. 71-91, 2011.
[3] P. Ferrara, et al., "Image forgery localization via fine-grained analysis of CFA artifacts," in IEEE Transactions on Information Forensics and Security, vol. 7, no. 5, pp. 1566-1577, 2012.
[4] H. Farid, S. Lyu, "Higher-order wavelet statistics and their application to digital forensics," in 2003 Conference on computer vision and pattern recognition workshop, vol. 8, pp. 94, 2003.
[5] A. Popescu, H. Farid, "Statistical tools for digital forensics," in International workshop on information hiding, pp. 128-147, 2004.
[6] B. Mahdian, S. Saic, "Using noise inconsistencies for blind image forensics," in Image and vision computing, vol. 27, no. 10, pp. 1497-1503, 2009.
[7] D. Cozzolino, G. Poggi, L. Verdoliva, "Splicebuster: A new blind image splicing detector," in 2015 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1-6, 2015.
[8] T. Bianchi, A. Piva, "Image forgery localization via block-grained analysis of JPEG artifacts," in IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 1003-1017, 2012.
[9] C. Iakovidou, et al., "Content-aware detection of JPEG grid inconsistencies for intuitive image forensics," in Journal of Visual Communication and Image Representation, vol. 54, pp. 155-170, 2018.
[10] X. Bi, et al., "RRU-Net: The ringed residual U-Net for image splicing forgery detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0, 2019.
[11] Y. Wu, W. AbdAlmageed, P. Natarajan, "Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features," in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 9543-9552, 2019.
[12] M. Huh, et al., "Fighting fake news: Image splice detection via learned self-consistency," in Proceedings of the European conference on computer vision (ECCV), pp. 101-117, 2018.
[13] A. Rossler, et al., "Faceforensics++: Learning to detect manipulated facial images," in Proceedings of the IEEE/CVF international conference on computer vision, pp. 1-11, 2019.
[14] H. Nguyen, J. Yamagishi, I. Echizen, "Capsule-forensics: Using capsule networks to detect forged images and videos," in ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2307-2311, 2019.
[15] P. Zhou, et al., "Two-stream neural networks for tampered face detection," in 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 1831-1839, 2017.
[16] B. Yu, et al., "Frequency-aware spatiotemporal transformers for video inpainting detection," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8188-8197, 2021.
[17] H. Liu, et al., "Spatial-phase shallow learning: rethinking face forgery detection in frequency domain," in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 772-781, 2021.
[18] X. Yang, Y. Li, S. Lyu, "Exposing deep fakes using inconsistent head poses," in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8261-8265, 2019.
[19] P. Gronquist, et al., "Efficient Temporally-Aware DeepFake Detection using H. 264 Motion Vectors," in arXiv preprint arXiv:2311.10788, 2023.
[20] D. Güera, E. Delp, "Deepfake video detection using recurrent neural networks," in 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS), pp. 1-6, 2018.
[21] D. Wodajo, S. Atnafu. "Deepfake video detection using convolutional vision transformer," in arXiv preprint arXiv:2102.11126, 2021.
[22] A. Haliassos, et al., "Lips don′t lie: A generalisable and robust approach to face forgery detection," in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 5039-5049, 2021.
[23] C. Feng, Z. Chen, A. Owens, "Self-supervised video forensics by audio-visual anomaly detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10491-10503, 2023.
[24] K. Zhang, et al., "Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising," in IEEE transactions on image processing, vol. 26, no. 7, pp. 3142-3155, 2017.
[25] J. Bromley, et al., "Signature verification using a" siamese" time delay neural network," in Advances in neural information processing systems, vol. 6, 1993.
[26] S. Chopra, R. Hadsell, Y. LeCun, "Learning a similarity metric discriminatively, with application to face verification," in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR′05), pp. 539-546, 2005.
[27] Q. Wang, et al., "Fast online object tracking and segmentation: A unifying approach," in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 1328-1338, 2019.
[28] J. Mueller, A. Thyagarajan, " Siamese Recurrent Architectures for Learning Sentence Similarity," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1, 2016.
[29] R. Hadsell, S. Chopra, Y. LeCun, " Dimensionality Reduction by Learning an Invariant Mapping," in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR′06), vol. 2, pp. 1735-1742, 2006.
[30] C. Rother, V. Kolmogorov, A. Blake, "" GrabCut" interactive foreground extraction using iterated graph cuts," in ACM transactions on graphics (TOG), vol. 23, no. 3, pp. 309-314, 2004.
[31] K. Zhang, et al., "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," in IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
[32] T. Gloe, R. Böhme, " The ′Dresden Image Database′ for benchmarking digital image forensics", in Proceedings of the 2010 ACM Symposium on Applied Computing, pp. 1584-1590, 2010.
[33] C. Galdi, F. Hartung, J. Dugelay, "Videos versus still images: Asymmetric sensor pattern noise comparison on mobile phones" in Electronic Imaging, vol. 29, pp 100-103, 2017.
[34] D. Shullani, et al., "Vision: a video and image dataset for source identification," in EURASIP Journal on Information Security, vol. 2017, pp. 1-16, 2017.
[35] T. De Carvalho, et al., "Exposing digital image forgeries by illumination color classification," in IEEE Transactions on Information Forensics and Security, vol. 8, no. 7, pp. 1182-1194, 2013.
[36] P. Jaccard, "The distribution of the flora in the alpine zone, "in New Phytologist, vol. 11, no. 2, pp. 37-50, 1912.
[37] P. Su, B. Huang, T. Kuo, "UFCC: A Unified Forensic Approach to Locating Tampered Areas in Still Images and Detecting Deepfake Videos by Evaluating Content Consistency," in Electronics, vol. 13, no. 4, pp. 804, 2024.
[38] T. Bianchi, A. Piva. "Image forgery localization via block-grained analysis of JPEG artifacts," in IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 1003-1017, 2012.
[39] S. Ye, Q. Sun, E. Chang, "Detecting Digital Image Forgeries by Measuring Inconsistencies of Blocking Artifact," in 2007 IEEE International Conference on Multimedia and Expo, pp. 12-15, 2007.
[40] B. Mahdian, S. Saic, "Using noise inconsistencies for blind image forensics," in Image and Vision Computing, vol. 27, no. 10, pp. 1497-1503, 2009.
[41] R. Salloum, Y. Ren, C. Kuo, "Image Splicing Localization using a Multi-task Fully Convolutional Network (MFCN)," in Journal of Visual Communication and Image Representation, vol. 51, pp. 201-209, 2018.
[42] H. Ding, et al., "DCU-Net: a dual-channel U-shaped network for image splicing forgery detection," in Neural Comput & Applic 35, pp. 5015–5031, 2023.
[43] Y. Zhang, et al., "Multi-Task SE-Network for Image Splicing Localization," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 7, pp. 4828-4840, 2022.
[44] F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800-1807, 2017.
[45] J. Fridrich, J. Kodovsky, "Rich Models for Steganalysis of Digital Images," in IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 868-882, 2012.
[46] D. Cozzolino, G. Poggi, L. Verdoliva, "Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection," in Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, pp. 159-164, 2017.
[47] B. Bayar, M. Stamm, "A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer," in Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, pp. 5–10, 2016.
[48] N. Rahmouni, et al., "Distinguishing computer graphics from natural images using convolution neural networks," in 2017 IEEE Workshop on Information Forensics and Security (WIFS), pp. 1-6, 2017.
[49] D. Afchar, et al., "MesoNet: a Compact Facial Video Forgery Detection Network," in 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1-7, 2018.
[50] R. Venkatesan, et al., "Robust image hashing," in Proceedings 2000 International Conference on Image Processing, vol. 3, pp. 664-666, 2000.
[51] R. Datta, et al., "Image retrieval: Ideas, influences, and trends of the new age," in ACM Comput. Surv., vol. 40, no. 2, pp. 60, 2008.
[52] M. Chen, et al., "Determining Image Origin and Integrity Using Sensor Noise," in IEEE Transactions on Information Forensics and Security, vol. 3, no. 1, pp. 74-90, 2008.
[53] Corel, Corel_Professional_Photos_Collection_1994, Internet Archive Python library 1.9.0, 1994, https://archive.org/details/Corel_Professional_Photos_Collection_1994.

指導教授

蘇柏齊(Po-Chyi Su)

審核日期

2024-8-19

推文