人工智慧應用於玻璃瑕疵檢測

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：42

、訪客IP：3.139.80.172

姓名

林楷裕(Kai-Yu Lin) 查詢紙本館藏

畢業系所

機械工程學系

論文名稱

人工智慧應用於玻璃瑕疵檢測
(Application of Artificial Intelligence to Defect Detection in Glas)

相關論文

★ 晶圓針測參數實驗與模擬分析	★ 車銑複合加工機床面結構最佳化設計
★ 精密空調冷凝器軸流風扇葉片結構分析	★ 第四代雙倍資料率同步動態隨機存取記憶體連接器應力與最佳化分析
★ PCB電性測試針盤最佳鑽孔加工條件分析	★ 鋰-鋁基及鋰-氮基複合儲氫材料之製程開發及研究
★ 合金元素(錳與鋁)與球磨處理對Mg2Ni型儲氫合金放電容量與循環壽命之影響	★ 鍶改良劑、旋壓成型及熱處理對A356鋁合金磨耗腐蝕性質之影響
★ 核電廠元件疲勞壽命模擬分析	★ 可撓式OLED封裝薄膜和ITO薄膜彎曲行為分析
★ MOCVD玻璃承載盤溫度場分析	★ 不同環境下之沃斯回火球墨鑄鐵疲勞裂縫成長行為
★ 不同環境下之Custom 450不銹鋼腐蝕疲勞性質研究	★ AISI 347不銹鋼腐蝕疲勞行為
★ 環境因素對沃斯回火球墨鑄鐵高週疲勞之影響	★ AISI 347不銹鋼在不同應力比及頻率下之腐蝕疲勞行為

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2027-8-1以後開放)

摘要(中)

自動光學檢測(AOI)是一個相當成熟且被廣泛使用在產品檢測上的技術，例如印刷電路板與平面顯示器。大多數的AOI機台是使用傳統影像處理來進行演算法的設計，這會造成AOI在一些產業上，仍然無法取代人工目檢的現象。以玻璃產品為例，實際使用AOI機台檢測會發現，粉塵與棉絮等非瑕疵造成系統大量的誤判。本研究乃針對玻璃產品的檢測系統進行探討，導入人工智慧技術於AOI來克服此困難，以期能在大小為 85 mm × 53 mm × 2 mm的玻璃試片上，檢測出刮傷與崩邊瑕疵。
在取像系統上，本研究針對產品與瑕疵的特性，進行取像方式的優化。透過使用隧道燈搭配暗場的取像效果，確保取像系統能夠清楚地捕捉瑕疵特徵。在提升影像品質上，則使用治具與吸光黑布，並成功降低影像中的雜訊。
在影像處理系統中，首先挑選5個YOLOv4相關的物件偵測模型進行訓練與比較，分別是YOLOv4、YOLOv4-tiny、YOLOv4-CSP、YOLOv4-P5與YOLOv4-P6。在自製資料集上，是以一套自定義的標註流程進行瑕疵標註，以確保標註的品質。本研究使用5-fold交叉驗證的方式來挑選最適合的使用模型。驗證結果顯示，YOLOv4有最高的準確度並且在推論影像大小僅為672 × 672像素的情況下也能維持良好的模型表現，因此將YOLOv4作為本研究的應用模型。在模型優化方面，使用兩種方式進行YOLOv4的優化，分別是微調與錨框的優化。後者成功提升YOLOv4的準確度，得到此最佳模型並以測試集進行測試後，分別得到精確率98.4%、召回率91.7%與mAP值94.52%。此錨框優化模型能夠良好地排除非瑕疵的干擾，並且能夠檢測僅0.05 mm寬度的刮傷與0.1 mm的崩邊瑕疵，更能夠偵測出僅以3個灰階值呈現的不明顯刮傷。另外，也透過使用此錨框優化模型與5-fold交叉驗證中選用的YOLOv4模型進行實際獨立全尺寸玻璃檢測，展示了兩者模型都能夠正確地檢測出在人工目檢中未被找到的瑕疵。比較以上兩者模型，錨框優化模型能夠準確地進行瑕疵檢測並且有效排除非瑕疵干擾，而原始選用的YOLOv4模型則是能夠有效地在最短時間內完成檢測。

摘要(英)

Automated optical inspection (AOI) is widely used in various industries, such as printed circuit board and flat panel display. In a traditional AOI system, digital image processing is applied to design an algorithm for the inspection. However, because of the limitation of AOI, manual visual inspection is still irreplaceable in certain cases, e.g. in glass manufacturing. Pollutions like dust particles and batting could reduce the performance of AOI system for glass. To overcome that, this study introduced an artificial intelligent technique into the AOI system for glass inspection. The inspection targets included two types of glass defects, namely scratch and chip. The goal was to find such defects on the glass specimens with dimensions of 85 mm × 53 mm × 2 mm.
In this study, the image acquisition system was optimized to capture the features of defects with the most suitable light source. For a better image quality, a jig and a black cloth were used to reduce the noise on the image. Such an image acquisition system could provide a high-quality image with defects, so that the image processing algorithm could analyze the information of defects on the specimens and detect defects.
In the image processing algorithm, 5 YOLOv4 variants were considered for the inspection application, namely YOLOv4, YOLOv4-tiny, YOLOv4-CSP, YOLOv4-P5, and YOLOv4-P6. The custom datasets were built to train these models. In order to have an objective annotation, it followed a procedure to label the custom datasets. After that, 5-fold cross validation was applied to compare each model’s performance. According to the validation results, YOLOv4 was selected for the application in this study as it had the highest accuracy and the ability of maintaining a great accuracy when the input image was resized to only 672 × 672 pixels. Then, two approaches were used to optimize the YOLOv4 model, namely fine-tuning and anchor box optimization. The latter method successfully improved the accuracy of YOLOv4. The most accurate model was the YOLOv4 optimized by anchor box and trained with the input size of 960 × 960 pixels. It had a precision of 98.4%, a recall of 91.7%, and mAP of 94.52% in the test dataset. Moreover, it could effectively exclude non-defect objects. With these results, the developed inspection system could detect defects as small as a scratch of 0.05-mm width and a chip of 0.1 mm. Furthermore, it could also detect an unclear scratch with only three-gray-level difference. Finally, the originally selected YOLOv4 model and its modification by anchor box optimization were applied to inspect two independent specimens of full-size images. The inspection results showed that both models successfully found defects that were not inspected by manual visual inspection. In comparison of the two models, the YOLOv4 model optimized by anchor box was more effective to detect defect and exclude non-defect objects, while the originally selected model of YOLOv4 was more efficient to inspect an entire specimen with a shorter time.

關鍵字(中)

★ 自動光學檢測
★ 人工智慧
★ 玻璃瑕疵檢測
★ YOLOv4

關鍵字(英)

論文目次

ABSTRACT I
ACKNOWLEDGEMENTS V
TABLE OF CONTENTS VI
LIST OF TABLES VIII
LIST OF FIGURES IX
1. INTRODUCTION 1
1.1 Defect Detection by Automated Optical Inspection 1
1.2 Automated Optical Inspection with Artificial Intelligence 3
1.3 Machine Learning 6
1.4 Deep Learning 8
1.5 Purpose 13
2. EXPERIMENT AND METHOD 15
2.1 Image Acquisition System 15
2.2 Image Processing Algorithm 22
2.3 Custom Dataset 24
2.4 Training, Validation, and Testing 29
2.5 Evaluation 30
3. RESULTS AND DISCUSSION 33
3.1 Comparison of YOLOv4 Variants 33
3.1.1 YOLOv4 33
3.1.2 YOLOv4-tiny 40
3.1.3 YOLOv4-CSP 46
3.1.4 YOLOv4-P5 52
3.1.5 YOLOv4-P6 58
3.1.6 Selecting the applied model 64
3.2 Final Test of YOLOv4 65
3.2.1 Optimization 65
3.2.2 Calibration of evaluation 71
3.2.3 Test of YOLOv4 73
3.2.4 Increase in number of negative samples 97
4. CONCLUSIONS 105
5. FUTURE WORK 107
REFERENCES 108

參考文獻

1. M. Baygin, M. Karakose, A. Sarimaden, and E. Akin, “Machine Vision Based Defect Detection Approach Using Image Processing,” International Artificial Intelligence and Data Processing Symposium, pp.1-5, 2017.
2. S. -L. Chen and S.-T. Chou, “TFT-LCD Mura Defect Detection Using Wavelet and Cosine Transforms,” Journal of Advanced Mechanical Design Systems and Manufacturing, Vol. 2, No. 3, pp. 441-453, 2008.
3. Advantech, Achieving The Key Success of PCB Manufacturing：Automated Optical Inspection (AOI) Systems, https://www.advantech.com/resources/case-study/automated-optical-inspection-aoi-systems-for-pcb-manufacturing, accessed on February 18, 2022.
4. Market Prospects, What is Automatic Optical Inspection (AOI) Technology?, https://www.market-prospects.com/articles/what-is-aoi-technology, accessed on February 18, 2022.
5. W.-C. Wang, L.-B. Chen, W.-J. Chang, S.-L. Chen, and K. S.-M. Li, “A Machine Vision Based Automatic Optical Inspection System for Measuring Drilling Quality of Printed Circuit Boards,” IEEE Access, Vol. 5, pp. 10817-10833, 2016.
6. Wavelength Opto-Electronic, Automated Optical Inspection (AOI) Guide: All You Need to Know About an AOI Machine, https://wavelength-oe.com/blog/automated-optical-inspection-aoi-machine/, accessed on January 29, 2022.
7. Qualitas Technologies, Area Scan vs Line Scan: What Fits You Better?, https://qualitastech.com/area-scan-vs-line-scan-cameras/, accessed on March 11, 2022.
8. Cognex, Area Scan vs. Line Scan, https://www.cognex.com/what-is/machine-vision/system-types/area-scan-vs-line-scan, accessed on March 11, 2022.
9. Bestell, Area Scan vs. Line Scan Camera: The Difference, https://www.bestell.com.sg/area-scan-vs-line-scan-camera-the-difference/, accessed on March 11, 2022.
10. Adimec, Exploring Bright Field versus Dark Field Lighting for Your Inspection Application, https://www.adimec.com/exploring-bright-field-versus-dark-field-lighting-for-your-inspection-application/, accessed on March 13, 2022.
11. T.-Y. Shih, Development of a Prototype Machine for Removing Glue on Printed Circuit Boards by Automated Optical Inspection, M.S. Thesis, National Central University, 2021.
12. Instrumental, An overview of Automated Optical Inspection (AOI), https://instrumental.com/resources/quality/an-overview-of-automated-optical-inspection-aoi/, accessed on February 19, 2022.
13. AAEON AI, AI Vision in Automated Optical Inspection, https://www.aaeon.ai/tw/applications/detail/automated-optical-inspection, accessed on February 19, 2022.
14. H.-C. Liao, Z.-Y. Lim, Y.-X. Hu, and H.-W. Tseng, “Guidelines of Automated Optical Inspection (AOI) System Development,” IEEE 3rd International Conference on Signal and Image Processing, pp. 1-5, 2018.
15. Phase 1 Technology Corp., The Difference between Machine Vision & Computer Vision, https://www.phase1vision.com/blog/the-difference-between-machine-vision-and-computer-vision, accessed on February 19, 2022.
16. Analytics Insight, The 5 Most Amazing Computer Vision Techniques to Learn, https://www.analyticsinsight.net/the-5-most-amazing-computer-vision-techniques-to-learn/, accessed on February 19, 2022.
17. SAS, Artificial Intelligence, Machine Learning, Deep Learning and Beyond, https://www.sas.com/en_us/insights/articles/big-data/artificial-intelligence-machine-learning-deep-learning-and-beyond.html, accessed on February 12, 2022.
18. I. Abdel-Qader, O. Abudayyeh, and M. E. Kelly, “Analysis of Edge-Detection Techniques for Crack Identification in Bridge,” Journal of Computing in Civil Engineering, Vol. 17, pp. 255-263, 2003.
19. J. Saeedi, M. Dotta, A. Galli, A. Nasciuti, U. Maradia, M. Boccadoro, L. Maria Gambardella, and A. Giusti, “Measurement and Inspection of Electrical Discharge Machined Steel Surfaces Using Deep Neural Networks,” Machine Vision and Applications, Vol. 32, No. 21, pp. 1-15, 2020.
20. J. Li, Z. Su, J. Gengand, and Y. Yin, “Real-time Detection of Steel Strip Surface Defects Based on Improved YOLO Detection Network,” IFAC-PapersOnLine, Vol. 51, No. 21, pp. 76-81, 2018.
21. X. Peng, Y. Chen, W. Yu, Z. Zhou, and G. Sun, “An Online Defects Inspection Method for Float Glass Fabrication Based on Machine Vision,” The International Journal of Advanced Manufacturing Technology, Vol. 39, No. 11, pp. 1180-1189, 2008.
22. F. Adamo, F. Attivissimo, A. D. Nisio, and M. Savino, “An Online Defects Inspection System for Satin Glass Based on Machine Vision,” IEEE Instrumentation and Measurement Technology Conference, pp. 288-293, 2009.
23. F. Adamo, F. Attivissimo, A. D. Nisio, and M. Savino, “A Low-Cost Inspection System for Online Defects Assessment in Satin Glass,” Measurement, Vol. 42, No. 9, pp. 1304-1311, 2009.
24. F. Adamo, F. Attivissimo, A. D. Nisio, and M. Savino, “Calibration of an Inspection System for Online Quality Control of Satin Glass,” IEEE Transactions on Instrumentation and Measurement, Vol. 59, No. 5, pp. 1035-1046, 2010.
25. D. Li, L.-Q. Liang, and W.-J. Zhang, “Defect Inspection and Extraction of The Mobile Phone Cover Glass Based on The Principal Components Analysis,” The International Journal of Advanced Manufacturing Technology, Vol. 73, pp. 1605-1614, 2014.
26. Z. He and L. Sun, “Surface Defect Detection Method for Glass Substrate Using Improved Otsu Segmentation,” Applied Optics, Vol. 54, No. 33, pp. 9823-9830, 2015.
27. M. Chang, B.-C. Chen, J. L. Gabayno, and M.-F. Chen, “Development of an Optical Inspection Platform for Surface Defect Detection in Touch Panel Glass,” International Journal of Optomechatronics, Vol. 10, No. 2, pp. 63-72, 2016.
28. C. Jian, J. Gao, and Y. Ao, “Automatic Surface Defect Detection for Mobile Phone Screen Glass Based on Machine Vision,” Applied Soft Computing, Vol. 52, pp. 348-358, 2017.
29. J. Park, H. Riaz, H. Kim, and J. Kim, “Advanced Cover Glass Defect Detection and Classification Based on Multi-DNN Model,” Manufacturing Letters, Vol. 23, pp. 53-61, 2020.
30. M. U. M. Bhutta, S. Aslam, P. Yun, J. Jiao, and M. Liu, “Smart-Inspect: Micro Scale Localization and Classification of Smartphone Glass Defects for Industrial Automation,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2860-2865, 2020.
31. J. Jiang, P. Cao, Z. Lu, W. Lou, and Y. Yang, “Surface Defect Detection for Mobile Phone Back Glass Based on Symmetric Convolutional Neural Network Deep Learning,” Applied Sciences, Vol. 10, pp. 3621-3633, 2020.
32. V7, What is Machine Learning? The Ultimate Beginner′s Guide, https://www.v7labs.com/blog/machine-learning-guide, accessed on February 23, 2022.
33. A. Anastasiou, E. I. Zacharaki, D. Alexandropoulos, K. Moustakas, and N. A. Vainos, “Machine Learning Based Technique Towards Smart Laser Fabrication of CGH,” Microelectronic Engineering, Vol. 227, No. 111314, pp. 1-6, 2020.
34. K. P. Murphy, “Introduction,” Chapter 1 in Machine Learning: A Probabilistic Perspective, The MIT Press Ltd, Cambridge, Massachusetts, United States, 2012.
35. R. S. Sutton and A. G. Barto, “Reinforcement Learning,” Chapter 1.1 in Reinforcement Learning: An Introduction, The MIT Press Ltd, Cambridge, Massachusetts, United States, 2015.
36. Synopsys, What is Reinforcement Learning?, https://www.synopsys.com/ai/what-is-reinforcement-learning.html, accessed on March 27, 2022.
37. M. Beyeler, “Working with Data in OpenCV and Python,” Chapter 3 in Machine Learning for OpenCV: Intelligent Image Processing with Python, Packt Publishing Ltd, Birmingham, West Midlands, United Kingdom, 2017.
38. FreeCodeCamp, Machine Learning Tutorial – Feature Engineering and Feature Selection For Beginners, https://www.freecodecamp.org/news/feature-engineering-and-feature-selection-for-beginners/, accessed on April 1, 2022.
39. Machine Learning Mastery, Discover Feature Engineering, How to Engineer Features and How to Get Good at It, https://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/, accessed on April 1, 2022.
40. Towards Data Science, What is Feature Engineering — Importance, Tools and Techniques for Machine Learning, accessed on April 1, 2022.
41. Serokell, A Guide to Deep Learning and Neural Networks, https://serokell.io/blog/deep-learning-and-neural-network-guide, accessed on April 1, 2022.
42. Thinkwik, Insights Of The Machine Learning And The Deep Learning, https://blog.thinkwik.com/insights-of-the-machine-learning-and-the-deep-learning/, accessed on April 1, 2022.
43. Nvidia, What’s the Difference Between Artificial Intelligence, Machine Learning and Deep Learning?, https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/, accessed on February 21, 2022.
44. F. Rosenblatt, “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Psychological Review, Vol. 65, No. 6, pp 386-408, 1958.
45. QuantStart, Introduction to Artificial Neural Networks and the Perceptron, https://www.quantstart.com/articles/introduction-to-artificial-neural-networks-and-the-perceptron/, accessed on April 5, 2022.
46. Level Up Coding, “Training a Single Perceptron,” https://levelup.gitconnected.com/training-a-single-perceptron-405026d61f4b, accessed on April 16, 2022.
47. D. Graupe, “Deep Learning Neural Networks: Methodology and Scope,” Chapter 1 in Deep Learning Neural Networks: Design and Case Studies, World Scientific Publishing Co., Inc., Hackensack, New Jersey, United States, 2016.
48. Built In, “15 Deep Learning Applications You Need to Know,” https://builtin.com/artificial-intelligence/deep-learning-applications, accessed on April 15, 2022.
49. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation Applied to Handwritten Zip Code Recognition,” Neural Computation, Vol. 1, No. 4, pp. 541-551, 1989.
50. D. Graupe, “Deep Learning Convolutional Neural Network,” Chapter 5 in Deep Learning Neural Networks: Design and Case Studies, World Scientific Publishing Co. Ltd, University of Illinois, Chicago, United States, 2016.
51. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based Learning Applied to Document Recognition,” Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998.
52. 斎藤康毅, “深度學習,” Chapter 8 in Deep learning: 用Python進行深度學習的基礎理論實作, Gotop Information Inc., New Taipei City, Taiwan, 2017. (In Chinese)
53. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 580-587, 2014.
54. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
55. E. Shelhamer, J. Long, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 4, pp. 640-651, 2017.
56. K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” IEEE International Conference on Computer Vision, pp. 2980-2988, 2017.
57. J. Lee, “Killer Applications of Industrial AI,” Chapter 4 in Industrial AI: Applications with Sustainable Performance, Springer Nature Singapore Pte. Ltd., Downtown Core, Singapore, 2020.
58. A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection,” arXiv, 2004.10934, 2020.
59. C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” Journal of Big Data, Vol. 6, No. 60, pp. 1-48, 2019.
60. J. Nassar, V. Pavon-Harr, M. Bosch, and I. McCulloh, “Assessing Data Quality of Annotations with Krippendorff Alpha for Applications in Computer Vision,” AAAI Symposium 2019, pp. 1-9, 2019.
61. M. Al-Rawi and D. Karatzas, “On the Labeling Correctness in Computer Vision Datasets,” Interactive Adaptive Learning Co-located with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Vol. 2192, pp. 1-23, 2018.
62. N. M. Müller and K. Markert, “Identifying Mislabeled Instances in Classification Datasets,” International Joint Conference on Neural Networks, pp. 1-8, 2019.
63. T. Xiao, T. Xia, Y. Yang, C. Huang, and X. Wang, “Learning from Massive Noisy Labeled Data for Image Classification,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 2691-2699, 2015.
64. S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering, Vol. 22, No. 10, pp. 1345-1359, 2010.
65. A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung, S. Gelly, and N. Houlsby, “Big Transfer (BiT): General Visual Representation Learning,” arXiv, 1912.11370, 2020.
66. S. Raschka, “Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning,” arXiv, 1811.12808, 2020.
67. C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “Scaled-YOLOv4: Scaling Cross Stage Partial Network,” IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13024-13033, 2021.
68. A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung, S. Gelly, and N. Houlsby “Big Transfer (BiT): General Visual Representation Learning,” Computer Vision – ECCV 2020, pp. 491-507, 2020.
69. J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 6517-6525, 2017.
70. Github, Loss Fluctuation in YOLOv4, https://github.com/AlexeyAB/darknet/issues/5527, accessed on July 12, 2022.
71. Github, Fluctuation in Loss for Training YOLOv4 on 30 Classes, https://github.com/AlexeyAB/darknet/issues/3039, accessed on July 12, 2022.
72. Github, Huge Avg Loss When Using YOLOv4x Mish, https://github.com/AlexeyAB/darknet/issues/7606, accessed on July 12, 2022.
73. J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” arXiv, 1804.02767, 2018.
74. A. Araujo, W. Norris, and J. Sim, “Computing Receptive Fields of Convolutional Neural Networks,” Distill, Vol. 4, No. 11, e21, 2019.
75. Z. Tian, C. Shen, H. Chen, and T. He, “FCOS: A simple and strong anchor-free object detector,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, No. 4, pp. 1922-1933, 2020.

指導教授

林志光

審核日期

2022-8-24

推文