C2CL: Centroid-Concentrated Contrastive Learning on Fault Detection and Classification in Industry 4.0

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：64

、訪客IP：18.118.208.127

姓名

謝承紘(CHENG-HUNG HSIEH) 查詢紙本館藏

畢業系所

資訊管理學系

論文名稱

(C2CL: Centroid-Concentrated Contrastive Learning on Fault Detection and Classification in Industry 4.0)

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2029-7-1以後開放)

摘要(中)

隨著工業4.0革命改變傳統製造流程，製造業在生產過程中廣泛部署感測器以即時收集製程監控參數，旨在降低最終產品的瑕疵率。製造業重視故障檢測與分類（FDC）任務，以找出關鍵的製程錯誤原因。近年來，研究人員開始探索監督式對比學習（SCL），通過將原始數據轉換為具辨別性的特徵表示來優化模型，從而提高下游分類器的準確性。然而，類別不平衡的工業製程資料集、長序列製程資料的獨特性、以及SCL在處理特徵相似但屬於不同類別數據時的挑戰，導致模型訓練難以達到最佳效果。為了解決這一問題，我們提出了基於中心點集中特性的對比學習（C2CL）架構，在SCL的基礎上引入各類別樣本群的共同特徵作為附加資訊。實驗結果證實，基於C2CL架構訓練的編碼器能生成具類間分離和同類內集中特性的特徵表示。我們通過全面的實驗研究，證明了所提出的模型在所有評估指標上均優於目前最先進的基準模型，尤其在真實電子零組件供應商資料集上，F1-score指標提升了5.98%。

摘要(英)

With the revolution of Industry 4.0 transforming traditional manufacturing processes, sensors are widely deployed in the manufacturing industry to collect real-time process monitoring parameters, aiming to reduce the defect rate of the final product. The industry focuses on fault detection and classification (FDC) tasks to identify key process error causes. In recent years, researchers have explored supervised contrastive learning (SCL), optimizing models by transforming raw data into discriminative feature representations, thereby improving downstream classifier accuracy. However, issues such as class imbalance in industrial process datasets, the unique nature of long sequence process data, and the difficulty of SCL in handling similar features that belong to different classes hinder optimal model training. To address this problem, we propose a Centroid-Concentrated Contrastive Learning (C2CL) framework, which introduces the common features of each class sample group as additional information based on SCL. Experimental results demonstrate that the encoder trained under the C2CL framework can generate feature representations with inter-class distraction and intra-class concentration. Through comprehensive experimental investigation, we prove that our proposed model outperforms the current state-of-the-art benchmark models across all evaluation metrics, especially achieving a 5.98% improvement in the F1-score on a real-world electronic component supplier dataset.

關鍵字(中)

★ 工業4.0
★ 深度學習
★ 監督式對比學習
★ 故障檢測與分類（FDC）

關鍵字(英)

★ Industry 4.0
★ Deep learning
★ Supervised Contrastive Learning
★ Fault Detection and Classification (FDC)

論文目次

摘要 i
Abstract ii
致謝 iii
Table of Contents iv
List of Figures iv
List of Tables vi
1. Introduction 1
2. Related Work 6
2.1 FDC Tasks in Semiconductor Manufacturing 6
2.2 Contrastive Learning Variants in FDC tasks 11
3. Proposed Model: C2CL 16
3.1 Data augmentation strategy 17
3.2 Centroid-Concentrated Contrastive Learning (C2CL) 18
3.3 Classification Model for Downstream FDC Tasks 22
4. Experiments and Evaluation 23
4.1 Evaluation Metric and Baseline Models 25
4.2 Performance Comparison 28
4.3 Effectiveness of Centroid Loss and Centroid-based Concentration Loss 32
4.4 Analysis of α and β Coefficient Configurations 33
4.5 Ablation Study 35
4.6 Sensitivity Analysis on Parameter Setting 38
4.7 Case Study 43
5. Conclusion 48
Reference 49

參考文獻

[1] H. Lasi, P. Fettke, H.-G. Kemper, T. Feld, and M. Hoffmann, "Industry 4.0," Business & Information Systems Engineering, vol. 6, no. 4, pp. 239-242, 2014.
[2] M. Ghobakhloo, "Industry 4.0, digitization, and opportunities for sustainability," Journal of Cleaner Production, vol. 252, p. 119869, 2020.
[3] B. Goodlin, D. S. Boning, H. H. Sawin, and B. M. Wise, "Simultaneous Fault Detection and Classification for Semiconductor Manufacturing Tools," Journal of The Electrochemical Society, vol. 150, 2003.
[4] Q. P. He and J. Wang, "Fault Detection Using the k-Nearest Neighbor Rule for Semiconductor Manufacturing Processes," IEEE Transactions on Semiconductor Manufacturing, vol. 20, no. 4, pp. 345-354, 2007.
[5] Q. P. He and J. Wang, "Principal component based k-nearest-neighbor rule for semiconductor process fault detection," in Proceedings of 2008 American Control Conference, pp. 1606-1611, 2008.
[6] G. Verdier and A. Ferreira, "Adaptive Mahalanobis Distance and $k$ -Nearest Neighbor Rule for Fault Detection in Semiconductor Manufacturing," IEEE Transactions on Semiconductor Manufacturing, vol. 24, no. 1, pp. 59-68, 2011.
[7] H. J. Chang, D. S. Song, P. J. Kim, and J. Y. Choi, "Spatiotemporal Pattern Modeling for Fault Detection and Classification in Semiconductor Manufacturing," IEEE Transactions on Semiconductor Manufacturing, vol. 25, no. 1, pp. 72-82, 2012.
[8] C.-F. Chien, C.-Y. Hsu, and P.-N. Chen, "Semiconductor fault detection and classification for yield enhancement and manufacturing intelligence," Flexible Services and Manufacturing Journal, vol. 25, no. 3, pp. 367-388, 2013.
[9] J. Yu, "Fault Detection Using Principal Components-Based Gaussian Mixture Model for Semiconductor Manufacturing Processes," IEEE Transactions on Semiconductor Manufacturing, vol. 24, no. 3, pp. 432-444, 2011.
[10] Z. Zhou, C. Wen, and C. Yang, "Fault Detection Using Random Projections and k-Nearest Neighbor Rule for Semiconductor Manufacturing Processes," IEEE Transactions on Semiconductor Manufacturing, vol. 28, no. 1, pp. 70-79, 2015.
[11] T. Lee and C. O. Kim, "Statistical Comparison of Fault Detection Models for Semiconductor Manufacturing Processes," IEEE Transactions on Semiconductor Manufacturing, vol. 28, no. 1, pp. 80-91, 2015.
[12] C. I. Lang, F. K. Sun, B. Lawler, J. Dillon, A. A. Dujaili, J. Ruth, P. Cardillo, P. Alfred, A. Bowers, A. Mckiernan, and D. S. Boning, "One Class Process Anomaly Detection Using Kernel Density Estimation Methods," IEEE Transactions on Semiconductor Manufacturing, vol. 35, no. 3, pp. 457-469, 2022.
[13] E. Kim, S. Cho, B. Lee, and M. Cho, "Fault Detection and Diagnosis Using Self-Attentive Convolutional Neural Networks for Variable-Length Sensor Data in Semiconductor Manufacturing," IEEE Transactions on Semiconductor Manufacturing, vol. 32, no. 3, pp. 302-309, 2019.
[14] T. Lee, K. B. Lee, and C. O. Kim, "Performance of Machine Learning Algorithms for Class-Imbalanced Process Fault Detection Problems," IEEE Transactions on Semiconductor Manufacturing, vol. 29, no. 4, pp. 436-445, 2016.
[15] S. K. S. Fan, D. M. Tsai, and P. C. Yeh, "Effective Variational-Autoencoder-Based Generative Models for Highly Imbalanced Fault Detection Data in Semiconductor Manufacturing," IEEE Transactions on Semiconductor Manufacturing, vol. 36, no. 2, pp. 205-214, 2023.
[16] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.
[17] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
[18] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, "Albert: A lite bert for self-supervised learning of language representations," arXiv preprint arXiv:1909.11942, 2019.
[19] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, and S. Gelly, "An image is worth 16x16 words: Transformers for image recognition at scale," arXiv preprint arXiv:2010.11929, 2020.
[20] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, "Swin transformer: Hierarchical vision transformer using shifted windows," in Proceedings of Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
[21] C. Zhang, J. Yella, Y. Huang, X. Qian, S. Petrov, A. Rzhetsky, and S. Bom, "Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word," in Proceedings of 2021 IEEE International Conference on Big Data (Big Data), pp. 1999-2008, 2021.
[22] W. Yu, X. Chen, G. Mao, Y. Wang, and Z. Wei, "A Neural Network Approach to Analyze FDC Data in Semiconductor Manufacture," in Proceedings of 2022 China Semiconductor Technology International Conference (CSTIC), pp. 1-3, 2022.
[23] C. Haolan, W. Mengmeng, W. Zhenghao, P. Yuchen, L. Donghuang, and L. Yadong, "Incipient Fault Detection of Power Distribution System based on Statistical Characteristics and Transformer Network," in Proceedings of 2022 Power System and Green Energy Conference (PSGEC), pp. 669-674, 2022.
[24] J. Yella, C. Zhang, S. Petrov, Y. Huang, X. Qian, A. A. Minai, and S. Bom, "Soft-Sensing ConFormer: A Curriculum Learning-based Convolutional Transformer," in Proceedings of 2021 IEEE International Conference on Big Data (Big Data), pp. 1990-1998, 2021.
[25] A. v. d. Oord, Y. Li, and O. Vinyals, "Representation Learning with Contrastive Predictive Coding," ArXiv, vol. abs/1807.03748, 2018.
[26] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, "Momentum contrast for unsupervised visual representation learning," in Proceedings of Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9738, 2020.
[27] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, "A simple framework for contrastive learning of visual representations," in Proceedings of International conference on machine learning, pp. 1597-1607, 2020.
[28] C. Wan, T. Zhang, Z. Xiong, and H. Ye, "Representation learning for fault diagnosis with contrastive predictive coding," in Proceedings of 2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes (SAFEPROCESS), pp. 1-5, 2021.
[29] J. Miao, Y. Liu, Q. Yin, B. Ju, G. Zhang, and H. Wang, "A Novel Soft Fault Detection and Diagnosis Method for a DC/DC Buck Converter Based on Contrastive Learning," IEEE Transactions on Power Electronics, 2023.
[30] Z. Bukhsh, "Contrastive Sensor Transformer for Predictive Maintenance of Industrial Assets," in Proceedings of ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3558-3562, 2022.
[31] A. Gao, F. Mei, J. Zheng, H. Sha, M. Guo, and Y. Xie, "Electricity theft detection based on contrastive learning and non-intrusive load monitoring," IEEE Transactions on Smart Grid, 2023.
[32] Y. Yao, J. Feng, and K. Li, "A Novel Fault Diagnosis Method based on Contrastive Learning with Multi-views of Data," in Proceedings of 2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes (SAFEPROCESS), pp. 1-5, 2021.
[33] J. Pöppelbaum, G. S. Chadha, and A. Schwung, "Contrastive learning based self-supervised time-series analysis," Applied Soft Computing, vol. 117, p. 108397, 2022.
[34] P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, "Supervised contrastive learning," Advances in neural information processing systems, vol. 33, pp. 18661-18673, 2020.
[35] P. Peng, X. Wang, J. Lu, Q. Li, S. Tao, Z. Wang, H. Wang, and H. Zhang, "Imbalanced Fault Diagnosis by Supervised Contrastive Learning," in Proceedings of 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 689-692, 2022.
[36] N.-Q. Dang, T.-T. Ho, T.-D. Vo-Nguyen, Y.-W. Youn, H.-S. Choi, and Y.-H. Kim, "Supervised Contrastive Learning for Fault Diagnosis Based on Phase-Resolved Partial Discharge in Gas-Insulated Switchgear," Energies, vol. 17, no. 1, p. 4, 2023.
[37] J. Zhang, J. Zou, Z. Su, J. Tang, Y. Kang, H. Xu, Z. Liu, and S. Fan, "A class-aware supervised contrastive learning framework for imbalanced fault diagnosis," Knowledge-Based Systems, vol. 252, p. 109437, 2022.
[38] Y. Chen, Z. Zhao, E. Kim, H. Liu, J. Xu, H. Min, and Y. Cui, "Wheel fault diagnosis model based on multichannel attention and supervised contrastive learning," Advances in Mechanical Engineering, vol. 13, no. 12, p. 16878140211067024, 2021.
[39] P. Peng, J. Lu, S. Tao, K. Ma, Y. Zhang, H. Wang, and H. Zhang, "Progressively balanced supervised contrastive representation learning for long-tailed fault diagnosis," IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-12, 2022.
[40] C. Hu, J. Wu, C. Sun, R. Yan, and X. Chen, "Robust Supervised Contrastive Learning for Fault Diagnosis under Different Noises and Conditions," in Proceedings of 2021 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence (ICSMD), pp. 1-6, 2021.
[41] X. Yang, H. He, and D. Liu, "Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems," IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 9, pp. 1866-1878, 2017.
[42] A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, "A survey on contrastive self-supervised learning," Technologies, vol. 9, no. 1, p. 2, 2020.
[43] Q. Wen, L. Sun, F. Yang, X. Song, J. Gao, X. Wang, and H. Xu, "Time series data augmentation for deep learning: A survey," arXiv preprint arXiv:2002.12478, 2020.
[44] J. Huang, Y. Song, H. Tan, and W. Zhang, "Few-shot learning for Defect Detection of Industrial Components," in Proceedings of TENCON 2022 - 2022 IEEE Region 10 Conference (TENCON), pp. 1-4, 2022.
[45] H. Wang, X. Yu, D. Chen, and S. Deng, "Few-Shot Object Detection Based on Contrastive Learning," in Proceedings of 2023 6th International Conference on Intelligent Robotics and Control Engineering (IRCE), pp. 236-242, 2023.
[46] Y. Wen, K. Zhang, Z. Li, and Y. Qiao, "A discriminative feature learning approach for deep face recognition," in Proceedings of Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14, pp. 499-515, 2016.
[47] C. A. Rieth, B. D. Amsel, R. Tran, and M. B. Cook. Additional Tennessee Eastman Process Simulation Data for Anomaly Detection Evaluation, Harvard Dataverse, 2017. Available: https://doi.org/10.7910/DVN/6C3JR1
[48] I. D. a. K. Katser, Vyacheslav O. Skoltech Anomaly Benchmark (SKAB), Kaggle, 2020. Available: https://www.kaggle.com/dsv/1693952
[49] L. E. Peterson, "K-nearest neighbor," Scholarpedia, vol. 4, no. 2, p. 1883, 2009.
[50] J. J. Downs and E. F. Vogel, "A plant-wide industrial process control problem," Computers & Chemical Engineering, vol. 17, no. 3, pp. 245-255, 1993.
[51] C. Yang, A Novel Deep Parallel Time-series Relation Network for Fault Diagnosis. 2021.
[52] L. H. Chiang, B. Jiang, X. Zhu, D. Huang, and R. D. Braatz, "Diagnosis of multiple and unknown faults using the causal map and multivariate statistics," Journal of Process Control, vol. 28, pp. 27-39, 2015.
[53] C. Jing, X. Gao, X. Zhu, and S. Lang, "Fault classification on Tennessee Eastman process: PCA and SVM," in Proceedings of 2014 International Conference on Mechatronics and Control (ICMC), pp. 2194-2197, 2014.
[54] D. Xie and L. Bai, "A Hierarchical Deep Neural Network for Fault Diagnosis on Tennessee-Eastman Process," in Proceedings of 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 745-748, 2015.
[55] R. Verma, R. Yerolla, and C. S. Besta, "Deep Learning-based Fault Detection in the Tennessee Eastman Process," in Proceedings of 2022 Second International Conference on Artificial Intelligence and Smart Energy (ICAIS), pp. 228-233, 2022.

指導教授

陳以錚陳振明

審核日期

2024-7-17

推文