以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：131

、訪客IP：18.222.117.136

姓名	林碩彥(Shuo-Yen Lin)  查詢紙本館藏	畢業系所	資訊工程學系
論文名稱	基於擴散模型的自然對抗補丁生成 (Diffusion to Confusion: Naturalistic Adversarial Patch Generation Based on Diffusion Model for Object Detector)
檔案	[Endnote RIS 格式]    [Bibtex 格式]    [相關文章]   [文章引用]   [完整記錄]   [館藏目錄]   至系統瀏覽論文 (2025-7-4以後開放)
摘要(中)	為了保障個人隱私資料免受不法份子惡意使用物件偵測器進行監控，近年來已經有許多物理對抗補丁生成方法被提出。然而，這些方法往往需要進行大量的超參數調整，並且必須在達到足夠的攻擊效果同時不被他人察覺。因此，生成外觀令人滿意的補丁圖像仍然是一個具有挑戰性的問題。為了解決這個問題，本研究提出了一種基於擴散模型（Diffusion Model）的新型自然對抗補丁生成方法。通過在自然圖像上預訓練的擴散模型中採樣最佳圖像，我們可以穩健地製作出高品質且外觀自然的對抗補丁，而避免其他深度生成模型所遇到的嚴重模式崩潰問題。據我們所知，本研究是第一個針對物件偵測器提出基於擴散模型的物理對抗性補丁生成方法。此外，通過廣泛的定量、定性和主觀實驗，我們發現相比於其他最先進的補丁生成方法，我們的方法可以有效地生成品質更好、更自然的對抗補丁，同時實現出色的攻擊性能
摘要(英)	Numerous physical adversarial patch generation methods have been proposed to protect personal privacy from malicious monitoring using object detectors. However, these methods often fall short of generating satisfactory patch images in terms of both stealthiness and attack performance without extensive hyperparameter tuning. To address this issue, we propose a novel naturalistic adversarial patch generation method based on diffusion models (DM). By sampling the optimal image from a DM model pre-trained on natural images, we can craft high-quality and naturalistic physical adversarial patches in a stable manner, without suffering from the serious mode collapse problems that plague other deep generative models. To the best of our knowledge, we are the first to propose a DM-based naturalistic adversarial patch generation method for object detectors. Extensive quantitative, qualitative, and subjective experiments demonstrate that our approach is effective in generating better-quality and more naturalistic adversarial patches while achieving acceptable attack performance compared to other state-of-the-art patch generation methods. Additionally, we show various generation trade-offs under different conditions
關鍵字(中)	★ 擴散模型 ★ 對抗補丁 ★ 對抗攻擊 ★ 物件偵測 ★ 攻擊物件偵測器 ★ 深度生成模型	關鍵字(英)	★ Diffusion Model ★ Adversarial Patch ★ Adversarial Attack ★ Object Detection ★ Attack Object Detector ★ Deep Generative Model
論文目次	摘要 i Abstract ii Contents iii 1 Introduction 1 2 Related Work 4 2.1 Digital Adversarial Example 4 2.2 Physical Adversarial Example 4 2.3 Naturalistic Adversarial Patch 5 3 Background 6 3.1 Diffusion Models 6 3.2 Speed up Generation Process 7 3.3 Diffusion on Low-Dimensional Latent Space 7 4 Method 8 4.1 Naturalistic Adversarial Patch Generation 9 4.1.1 Patch Generation 9 4.1.2 Scene Rendering 9 4.2 Loss Function 10 4.3 Maintaining Naturalness 10 4.3.1 Using Text Condition 10 4.3.2 Search Strategy 11 5 Experiment Result 12 5.1 Implementation Details 12 5.2 Victim Models 12 5.3 Dataset 13 5.4 Evaluation Setup 13 5.5 Cross-Model Generalizability 13 5.6 Attack Various Detectors 13 5.7 User Study 15 5.8 Cross-Dataset Evaluation 16 5.9 Attack in Physical World 16 5.10 Robustness to Existing Defenses 17 6 Ablation study 18 6.1 Diffusion Parameters 18 6.1.1 Noise Level 18 6.1.2 Step Size 18 6.1.3 Classifier-Free Guidance Scale 19 6.2 Text Condition 20 6.3 Different Pretrained Models 21 7 Conclusion 22 References 23 A Experiment Details 27 A.1 Optimization Hyperparameters 27 A.2 Samples from Datasets 27 A.3 Pretrained Models 27 A.4 Text Conditioning 30 A.5 Physical Experiments 30 B User study 33
參考文獻	[1] C. Chen, A. Seff, A. Kornhauser, and J. Xiao, “Deepdriving: Learning affordance for direct perception in autonomous driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2015, pp. 2722–2730. [2] S. M. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, and M. K. Khan, “Medical image analysis using convolutional neural networks: A review,” Journal of Medical Systems, vol. 42, pp. 1–13, 2018. [3] Y. Li, D. Tian, M.-C. Chang, X. Bian, and S. Lyu, “Robust adversarial perturbation on deep proposal-based models,” in Proceedings of the British Machine Vision Conference, 2018, p. 231. [4] C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille, “Adversarial examples for semanticsegmentation and object detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2017, pp. 1369–1378. [5] J. Li, F. Schmidt, and Z. Kolter, “Adversarial camera stickers: A physical camera-based attack on deep learning systems,” in International Conference on Machine Learning, 2019, pp. 3896–3904. [6] M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, “Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,” in Proceedings of the ACM Sigsac Conference on Computer and Communications Security, 2016, pp. 1528–1540. [7] S. Thys, W. Van Ranst, and T. Goedemé, “Fooling automated surveillance cameras: Adversarial patches to attack person detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019. [8] K. Xu, G. Zhang, S. Liu, et al., “Adversarial t-shirt! evading person detectors in a physical world,” in Proceedings of the European Conference on Computer Vision, 2020, pp. 665–681. [9] L. Huang, C. Gao, Y. Zhou, et al., “Universal physical camouflage attacks on object detectors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 720–729. [10] J. Tan, N. Ji, H. Xie, and X. Xiang, “Legitimate adversarial patches: Evading human eyes and detection models in the physical world,” in Proceedings of the ACM International Conference on Multimedia, 2021, pp. 5307–5315. [11] Y.-C.-T. Hu, B.-H. Kung, D. S. Tan, J.-C. Chen, K.-L. Hua, and W.-H. Cheng, “Naturalistic physical adversarial patch for object detectors,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 7848–7857. [12] I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., “Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014, pp. 2672–2680. [13] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, vol. 33, 2020. [14] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International Conference on Machine Learning, 2015, pp. 2256–2265. [15] C. Szegedy, W. Zaremba, I. Sutskever, et al., “Intriguing properties of neural networks,” in International Conference on Learning Representations, 2014. [16] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in Proceedings of the IEEE Symposium on Security and Privacy, 2017, pp. 39–57. [17] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint, 2014. [Online]. Available: http://arxiv.org/abs/1412.6980. [18] I. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in International Conference on Learning Representations, 2015. [19] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” in International Conference on Learning Representations, 2017. [20] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in International Conference on Learning Representations, 2018. [21] A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok, “Synthesizing robust adversarial examples,” in International Conference on Machine Learning, 2018, pp. 284–293. [22] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, “Adversarial patch,” in NeurIPS 2017 Workshop on Machine Learning and Computer Security, 2017. [23] S.-T. Chen, C. Cornelius, J. Martin, and D. H. Chau, “Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2019, pp. 52–68. [24] K. Eykholt, I. Evtimov, E. Fernandes, et al., “Robust physical-world attacks on deep learning visual classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 1625–1634. [25] A. Liu, X. Liu, J. Fan, et al., “Perceptual-sensitive gan for generating adversarial patches,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 1028–1035. [26] C. Sitawarin, A. N. Bhagoji, A. Mosenia, M. Chiang, and P. Mittal, “Darts: Deceiving autonomous cars with toxic signs,” arXiv preprint, 2018. [Online]. Available: http://arxiv.org/abs/1802.06430. [27] D. Song, K. Eykholt, I. Evtimov, et al., “Physical adversarial examples for object detectors,” in Proceedings of the USENIX Workshop on Offensive Technologies, 2018. [28] S. Komkov and A. Petiushko, “Advhat: Real-world adversarial attack on arcface face id system,” in Proceedings of the IEEE International Conference on Pattern Recognition, 2021, pp. 819–826. [29] M. Pautov, G. Melnikov, E. Kaziakhmedov, K. Kireev, and A. Petiushko, “On adversarial patches: Real-world attack on arcface-100 face recognition system,” in Proceedings of the International Multi-Conference on Engineering, Computer and Information Sciences, 2019, pp. 0391–0396. [30] Z. Wu, S.-N. Lim, L. S. Davis, and T. Goldstein, “Making an invisibility cloak: Real world adversarial attacks on object detectors,” in Proceedings of the European Conference on Computer Vision, 2020, pp. 1–17. [31] R. Duan, X. Ma, Y. Wang, J. Bailey, A. K. Qin, and Y. Yang, “Adversarial camouflage: Hiding physical-world attacks with natural styles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1000–1008. [32] J. Luo, T. Bai, and J. Zhao, “Generating adversarial yet inconspicuous patches with a single image (student abstract),” in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 15 837–15 838. [33] D. Kingma, T. Salimans, B. Poole, and J. Ho, “Variational diffusion models,” in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 21 696–21 707. [34] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in International Conference on Learning Representations, 2021. [35] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 684–10 695. [36] J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 7263–7271. [37] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint, 2018. [Online]. Available: http://arxiv.org/abs/1804.02767. [38] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint, 2020. [Online]. Available: http://arxiv.org/abs/2004.10934. [39] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in Neural Information Processing Systems, 2015. [40] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable DETR: deformable transformers for end-to-end object detection,” in International Conference on Learning Representations, 2021. [41] G. Jocher, A. Chaurasia, A. Stoken, et al., ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation, version v7.0, 2022. DOI: 10 . 5281 / zenodo . 7347926. [Online].Available: https://doi.org/10.5281/zenodo.7347926. [42] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” arXiv preprint, 2022. [Online]. Available: http://arxiv.org/abs/2207.02696. [43] T.-Y. Lin, M. Maire, S. Belongie, et al., “Microsoft coco: Common objects in context,” in Proceeding of the European Conference on Computer Vision, 2014, pp. 740–755. [44] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2005, pp. 886–893. [45] M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele, “2d human pose estimation: New benchmark and state of the art analysis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2014, pp. 3686–3693. [46] J. Liu, A. Levine, C. P. Lau, R. Chellappa, and S. Feizi, “Segment and complete: Defending object detectors against adversarial patch attacks with robust patch detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 14 973–14 982. [47] A. Radford, J. W. Kim, C. Hallacy, et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning, 2021, pp. 8748–8763. [48] J. Ho and T. Salimans, “Classifier-free diffusion guidance,” in NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
指導教授	王家慶 陳駿丞(Jia-Ching Wang Jun-Cheng Chen)	審核日期	2023-7-11
推文	facebook   plurk   twitter   funp   google   live   udn   HD   myshare   reddit   netvibes   friend   youpush   delicious   baidu
網路書籤	Google bookmarks   del.icio.us   hemidemi   myshare