基於視覺感知模型之深度偽造對抗性擾動

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：81

、訪客IP：18.222.161.123

姓名

王冠中(Kuan-Chung Wang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於視覺感知模型之深度偽造對抗性擾動
(Adversarial Perturbation against Deepfakes based on Visual Perceptual Model)

相關論文

★ 基於QT之跨平台無線心率分析系統實現	★ 網路電話之額外訊息傳輸機制
★ 針對與運動比賽精彩畫面相關串場效果之偵測	★ 植基於向量量化之視訊/影像內容驗證技術
★ 植基於串場效果偵測與內容分析之棒球比賽精華擷取系統	★ 以視覺特徵擷取為基礎之影像視訊內容認證技術
★ 使用動態背景補償以偵測與追蹤移動監控畫面之前景物	★ 應用於H.264/AVC視訊內容認證之適應式數位浮水印
★ 棒球比賽精華片段擷取分類系統	★ 利用H.264/AVC特徵之多攝影機即時追蹤系統
★ 利用隱式型態模式之高速公路前車偵測機制	★ 基於時間域與空間域特徵擷取之影片複製偵測機制
★ 結合數位浮水印與興趣區域位元率控制之車行視訊編碼	★ 應用於數位智權管理之H.264/AVC視訊加解密暨數位浮水印機制
★ 基於文字與主播偵測之新聞視訊分析系統	★ 植基於數位浮水印之H.264/AVC視訊內容驗證機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-1-11以後開放)

摘要(中)

深度偽造技術的出現對於數位視訊真實性帶來很大的威脅，近期許多研究針對深度偽造內容是否存在於視訊中發表相關的偵測與辨識方法，另也有研究學者提出在公開的影像中嵌入所謂對抗性浮水印，試圖使深偽模型所生成的竄改影像內容偏離預期結果，避免產生有效的竄改內容。現有的浮水印方法多於像素域中加入這種對抗性訊號，然而為了避免過強的浮水印訊號損及原影像畫質，無法在像素值施予較大幅度的改變。本研究提出於影像頻率域中嵌入對抗性浮水印，將影像轉換至亮度及色度空間後計算離散餘弦轉換(Discrete Cosine Transform, DCT)，透過Watson感知模型計算在不被人眼察覺下，確保DCT係數的修改低於可能的最大改變量，並依此決定浮水印在訓練階段時的修改步長。實驗結果顯示，所加入的高強度浮水印訊號確實能使深偽模型所生成的影像更容易發生嚴重失真，同時藉由計算影像畫質衡量來證實這樣的方法與像素值嵌入方法相比可有效降低對於原影像畫質的破壞。

摘要(英)

The emergence of Deepfakes poses a serious threat to the authenticity of digital videos. Recently, many studies have proposed methods for detecting and identifying the presence of Deepfakes in videos. On the other hand, some researchers adopted the approach of digital watermarking by embedding adversarial signals in public images to make the tampering results generated by Deepfake models deviate from their expected goals, so as to avoid producing effective falsified content. Most existing watermarking methods embedded such adversarial signals in the pixel domain. However, in order to prevent the quality of original image from being damaged by overly strong watermark signals, making large changes to the pixel values is not feasible. In this research, we propose to embed the adversarial watermark signals in the frequency domain of images. After converting the image from RGB color channels to YUV channels, the DCT (Discrete Cosine Transform) is applied on each channel. The Watson’s perception model is employed to determine the maximum possible change of DCT coefficients to ensure that the modification won’t be noticed by the human eyes. The perceptual mask is also used to determine the modification step size of the watermark in the training stage. The experimental results show that embedding such stronger watermarking signals can introduce more severe distortions on the image generated by the Deepfake models.

關鍵字(中)

★ 深度偽造
★ 對抗性浮水印
★ 深度學習

關鍵字(英)

★ Deepfakes
★ adversarial watermark
★ deep learning

論文目次

論文摘要 I
Abstract II
目錄 III
第一章緒論 1
1.1 研究動機 1
1.2 研究貢獻 2
1.3 論文架構 2
第二章相關研究 3
2.1 深度偽造(Deepfakes) 3
2.1.1 特徵修改 3
2.2 對抗攻擊 7
2.2.1 圖像攻擊 7
2.2.2 生成模型之對抗攻擊 8
2.3 視覺感知模型 12
2.4 資料集 13
第三章提出方法 14
3.1 對抗攻擊之浮水印 14
3.1.1 對單個模型之對抗攻擊 14
3.1.2 跨模型對抗攻擊 15
3.2 Watson感知模型 16
3.2.1 離散餘弦轉換 16
3.2.2 基於感知誤差之攻擊 18
3.3 跨模型步長修正 21
3.4 訓練流程 22
第四章實驗結果 24
4.1 開發環境 24
4.2 測試資料 24
4.2.1 驗證集 24
4.2.2 深度偽造模型 25
4.3 分數計算 26
4.3.1 原始影像質量 26
4.3.2 深度偽造破壞 27
4.4 成效 28
4.4.1 質量維護 28
4.4.2 Deepfakes輸出之破壞效果 29
第五章結論與未來展望 33
5.1 結論 33
5.2 未來展望 33
參考文獻 34

參考文獻

[1] Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C. C. "The deepfake detection challenge (dfdc) dataset." arXiv preprint arXiv:2006.07397 (2020).
[2] Deepfakes github. https://github.com/deepfakes/faceswap.
[3] FaceSwap. https://github.com/MarekKowalski/FaceSwap/
[4] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. "Generative adversarial networks." Communications of the ACM 63.11 (2020): 139-144.
[5] Tang, H., Xu, D., Sebe, N., & Yan, Y. "Attention-guided generative adversarial networks for unsupervised image-to-image translation." 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019.
[6] He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. "Attgan: Facial attribute editing by only changing what you want." IEEE transactions on image processing 28.11 (2019): 5464-5478.
[7] Choi, Y., Choi, M., Kim, M., Ha, J. W., Kim, S., & Choo, J. "Stargan: Unified generative adversarial networks for multi-domain image-to-image translation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
[8] Li, X., Zhang, S., Hu, J., Cao, L., Hong, X., Mao, X., ... & Ji, R. "Image-to-image translation via hierarchical style disentanglement." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
[9] Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. "Explaining and harnessing adversarial examples." arXiv preprint arXiv:1412.6572 (2014).
[10] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. "Going deeper with convolutions." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
[11] Ruiz, Nataniel, Sarah Adel Bargal, and Stan Sclaroff. "Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems." European Conference on Computer Vision. Springer, Cham, 2020.
[12] Watson, Andrew B. "DCT quantization matrices visually optimized for individual images." Human vision, visual processing, and digital display IV. Vol. 1913. SPIE, 1993.
[13] Liu, Z., Luo, P., Wang, X., & Tang, X. "Deep learning face attributes in the wild." Proceedings of the IEEE international conference on computer vision. 2015.
[14] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. "Towards deep learning models resistant to adversarial attacks." arXiv preprint arXiv:1706.06083 (2017).
[15] Huang, H., Wang, Y., Chen, Z., Zhang, Y., Li, Y., Tang, Z., ... & Ma, K. K. "Cmua-watermark: A cross-model universal adversarial watermark for combating deepfakes." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 1. 2022.
[16] Peterson, Heidi A., Albert J. Ahumada Jr, and Andrew B. Watson. "An Improved Detection Model for DCT Coefﬁcient Quantization." Human Vision and Electronic Imaging. Vol. 1913, 191-201. SPIE, 1993.
[17] Peterson, H. A., Peng, H., Morgan, J. H., & Pennebaker, W. B. "Quantization of color image components in the DCT domain." Human Vision, visual processing, and digital display II. Vol. 1453. SPIE, 1991.
[18] Ahumada Jr, Albert J., and Heidi A. Peterson. "Luminance-model-based DCT quantization for color image compression." Human vision, visual processing, and digital display III. Vol. 1666. SPIE, 1992.
[19] Huang, G. B., Mattar, M., Berg, T., & Learned-Miller, E. "Labeled faces in the wild: A database forstudying face recognition in unconstrained environments." Workshop on faces in′Real-Life′Images: detection, alignment, and recognition. 2008.
[20] Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. "The unreasonable effectiveness of deep features as a perceptual metric." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
[21] Kurakin, Alexey, Ian J. Goodfellow, and Samy Bengio. "Adversarial examples in the physical world." Artificial intelligence safety and security. Chapman and Hall/CRC, 2018. 99-112.
[22] Li, J., Ji, R., Liu, H., Hong, X., Gao, Y., & Tian, Q. "Universal perturbation attack against image retrieval." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.

指導教授

蘇柏齊(Po-Chyi Su)

審核日期

2023-1-11

推文