針對深度偽造生成影像之對抗性擾動訊號嵌入策略

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：152

、訪客IP：3.145.52.253

姓名

張友安(Yu-An Chang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

針對深度偽造生成影像之對抗性擾動訊號嵌入策略
(Effective Strategies of Adversarial Signal Embedding for Resisting Deepfakes Images)

相關論文

★ 基於QT之跨平台無線心率分析系統實現	★ 網路電話之額外訊息傳輸機制
★ 針對與運動比賽精彩畫面相關串場效果之偵測	★ 植基於向量量化之視訊/影像內容驗證技術
★ 植基於串場效果偵測與內容分析之棒球比賽精華擷取系統	★ 以視覺特徵擷取為基礎之影像視訊內容認證技術
★ 使用動態背景補償以偵測與追蹤移動監控畫面之前景物	★ 應用於H.264/AVC視訊內容認證之適應式數位浮水印
★ 棒球比賽精華片段擷取分類系統	★ 利用H.264/AVC特徵之多攝影機即時追蹤系統
★ 利用隱式型態模式之高速公路前車偵測機制	★ 基於時間域與空間域特徵擷取之影片複製偵測機制
★ 結合數位浮水印與興趣區域位元率控制之車行視訊編碼	★ 應用於數位智權管理之H.264/AVC視訊加解密暨數位浮水印機制
★ 基於文字與主播偵測之新聞視訊分析系統	★ 植基於數位浮水印之H.264/AVC視訊內容驗證機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2026-8-15以後開放)

摘要(中)

利用生成模型進行深度偽造的技術日益進步且易於使用，可能的應用包括將輸入的人物影像合成符合某種需求如特定表情與外觀的輸出影像，或者是將影像轉換為不同的風格的畫面。此類應用同時也帶來不少潛在隱憂。大多數生成模型影像包含人臉，但其來源可能觸及敏感議題或未經畫面人物的授權使用，如何防範影像的不當使用是值得關注的議題。
一種對人臉生成模型的反制方法是在影像中加入微小但不易察覺的擾動，藉此干預後續生成模型的運作。現存方法雖然讓加入擾動訊號的影像在生成模型的產出中產生內容破壞，但嵌入的擾動訊號卻容易造成影像明顯的失真，減少了實際運用的可行性。本研究提出結合視覺感知之最小可覺差(Just Noticeable Difference)與多種對抗性影像生成演算法的方式，產生與原圖更接近的擾動訊號嵌入影像，並探究不同的實作方式以確認對於生成模型的產出進行有效破壞。為了驗證擾動的適應性，我們亦測試反擾動攻擊，藉此比較對抗性擾動策略的優劣。實驗結果顯示，與現有方式限制最大像素值改變的方法相比，在保證對於目標生成模型的破壞效果下，我們基於最小可覺差的方法在影像品質的保持有更佳的表現。

摘要(英)

The technology for deepfakes using generative models is rapidly advancing and becoming increasingly accessible. Potential applications include synthesizing images of individuals that match specific requirements, such as certain expressions and appearances, or converting images into different styles. However, these applications also bring serious concerns. Most generative model outputs contain human faces, but their sources may involve sensitive issues or unauthorized use of individuals’ images. Preventing the misuse of such images is an important issue. One countermeasure against facial generative models is to introduce subtle but imperceptible perturbations into images to disrupt the subsequent operation of generative models. Existing methods, while causing content disruption in the outputs of generative models, often result in noticeable distortions in the images with embedded perturbations, reducing their practical usability. This study proposes a method that combines Just Noticeable Difference (JND) with various adversarial image generation strategies to produce perturbations that are closer to the original image. We also explore different implementation methods to ensure effective disruption of the generative model’s output. To validate the adaptability of the perturbations, we test against counter-perturbation attacks, comparing the effectiveness of different adversarial perturbation strategies. Experimental results show that, compared to existing methods that limit the maximum pixel value change, our JND-based approach provides better image quality preservation while ensuring effective disruption of the target generative model.

關鍵字(中)

★ 深度偽造
★ 視覺感知模型
★ GAN
★ 對抗性擾動
★ 深度學習

關鍵字(英)

論文目次

目錄
摘要 I
Abstract II
致謝 III
目錄 IV
圖目錄 VII
表目錄 IX
第一章、緒論 1
1.1. 研究動機 1
1.2. 研究貢獻 3
1.3. 論文架構 4
第二章、相關研究 5
2.1. 影像翻譯和深度偽造 5
2.2. 對抗性擾動演算法 7
2.2.1 Fast Gradient Sign Method (FGSM) 7
2.2.2 Iterative FGSM (I-FGSM) 9
2.2.3 Projected Gradient Descent (PGD) 10
2.3. 對抗性擾動之使用情境 11
2.4. 資料集 12
第三章、提出方法 13
3.1. 最小可覺差 14
3.1.1 Watson感知模型 (Watson Perceptual Model) 16
3.1.2 JPEG量化矩陣 19
3.2. 基於RGB影像的對抗性擾動 20
3.2.1 Jnd Limit FGSM (JL-FGSM) 22
3.2.2 JL-IFGSM & JL-PGD 24
3.3. 基於頻率域影像的對抗性擾動 26
3.3.1 Frequency Perturbations-FGSM (FP-FGSM) 28
3.3.2 FP-IFGSM & FP-PGD 29
第四章、實驗結果 31
4.1. 開發環境 31
4.2. 測試資料集 31
4.3. 指標評估 31
4.3.1 Peak Signal-to-Noise Ratio (PSNR) 32
4.3.2 Structural Similarity Index Measure (SSIM) 32
4.3.3 Learned Perceptual Image Patch Similarity (LPIPS) 33
4.4. 對抗性影像之品質及其經深偽模型之破壞效果比較 33
4.4.1 不同迭代次數之探討 33
4.4.2 不同Quality Factor之JPEG量化矩陣 35
4.5. 不同對抗性影像方法之品質及破壞效果 36
4.6. 對抗性影像及破壞效果之示例 38
4.6.1 基於RGB影像的對抗性擾動之示例 38
4.6.2 基於頻率域影像的對抗性擾動之示例 38
4.7. 對抗性影像攻擊之策略 40
4.7.1 基於影像處理的攻擊策略 40
4.7.2 基於JPEG量化的攻擊策略 41
4.7.3 基於深度學習的攻擊策略 43
第五章、結論與未來展望 45
5.1. 結論 45
5.2. 未來展望 45
參考文獻 46

參考文獻

[1] M. Zanardelli, F. Guerrini, R. Leonardi, N. Adami, "Image forgery detection: a survey of recent deep-learning approaches," in Multimedia Tools and Applications, vol. 82, no. 12, pp. 17521–17566, 2023.
[2] N. Ruiz, S. A. Bargal, S. Sclaroff, "Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems," in Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pp. 236–251, 2020.
[3] P. Isola, J.-Y. Zhu, T. Zhou, A. A. Efros, "Image-to-image translation with conditional adversarial networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134, 2017.
[4] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. Shi, "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690, 2017.
[5] J.-Y. Zhu, T. Park, P. Isola, A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," in Proceedings of the IEEE international conference on computer vision, pp. 2223–2232, 2017.
[6] Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, J. Choo, "Stargan: Unified generative adversarial networks for multi-domain image-to-image translation," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789–8797, 2018.
[7] I. J. Goodfellow, J. Shlens, C. Szegedy, "Explaining and harnessing adversarial examples," in arXiv preprint arXiv:1412.6572, 2014.
[8] A. Kurakin, I. Goodfellow, S. Bengio, "Adversarial machine learning at scale," in arXiv preprint arXiv:1611.01236, 2016.
[9] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, A. Vladu, "Towards deep learning models resistant to adversarial attacks," in arXiv preprint arXiv:1706.06083, 2017.
[10] T. Karras, T. Aila, S. Laine, J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation," in arXiv preprint arXiv:1710.10196, 2017.
[11] Z. Liu, P. Luo, X. Wang, X. Tang, "Deep learning face attributes in the wild," in Proceedings of the IEEE international conference on computer vision, pp. 3730–3738, 2015.
[12] A. B. Watson, "Image compression using the discrete cosine transform," in Mathematica journal, vol. 4, no. 1, pp. 81, 1994.
[13] A. B. Watson, "DCT quantization matrices visually optimized for individual images," in Human vision, visual processing, and digital display IV, pp. 202–216, 1993.
[14] I. Cox, M. Miller, J. Bloom, J. Fridrich, T. Kalker, "Digital watermarking and steganography," in Morgan kaufmann, 2007.
[15] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586–595, 2018.
[16] K. Simonyan, A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in arXiv preprint arXiv:1409.1556, 2014.
[17] A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, vol. 25, 2012.
[18] G. E. Hinton, R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," in science, vol. 313, no. 5786, pp. 504–507, 2006.
[19] A. v. d. Oord, O. Vinyals, K. Kavukcuoglu, "Neural discrete representation learning," in Advances in neural information processing systems, vol. 30, 2017.
[20] O. Ronneberger, P. Fischer, T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234–241, 2015.
[21] Z. Wang, X. Cun, J. Bao, W. Zhou, J. Liu, H. Li, "Uformer: A general u-shaped transformer for image restoration," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17683–17693, 2022.
[22] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, "Attention is all you need," in arXiv preprint arXiv:1706.03762, 2017.

指導教授

蘇柏齊

審核日期

2024-8-19

推文