結合深度學習與點雲分析之噴槍姿態辨識系統研究

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：17

、訪客IP：18.218.195.15

姓名

黃凱鴻(Kai-Hung Huang) 查詢紙本館藏

畢業系所

機械工程學系

論文名稱

結合深度學習與點雲分析之噴槍姿態辨識系統研究
(Development of a Spray Gun Pose Recognition System Integrating Deep Learning and Point Cloud Analysis)

相關論文

★ 晶圓針測參數實驗與模擬分析	★ 車銑複合加工機床面結構最佳化設計
★ 精密空調冷凝器軸流風扇葉片結構分析	★ 第四代雙倍資料率同步動態隨機存取記憶體連接器應力與最佳化分析
★ PCB電性測試針盤最佳鑽孔加工條件分析	★ 鋰-鋁基及鋰-氮基複合儲氫材料之製程開發及研究
★ 合金元素(錳與鋁)與球磨處理對Mg2Ni型儲氫合金放電容量與循環壽命之影響	★ 鍶改良劑、旋壓成型及熱處理對A356鋁合金磨耗腐蝕性質之影響
★ 核電廠元件疲勞壽命模擬分析	★ 可撓式OLED封裝薄膜和ITO薄膜彎曲行為分析
★ MOCVD玻璃承載盤溫度場分析	★ 不同環境下之沃斯回火球墨鑄鐵疲勞裂縫成長行為
★ 不同環境下之Custom 450不銹鋼腐蝕疲勞性質研究	★ AISI 347不銹鋼腐蝕疲勞行為
★ 環境因素對沃斯回火球墨鑄鐵高週疲勞之影響	★ AISI 347不銹鋼在不同應力比及頻率下之腐蝕疲勞行為

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2030-2-1以後開放)

摘要(中)

本研究目的在於開發一套基於深度學習與點雲分析的汽車噴漆噴槍姿態辨識系統，以解決傳統人工檢驗在噴漆作業效率低下與準確性不足的問題。此系統透過結合深度學習模型與 ZED 2 深度攝影機，以實現噴槍姿態與被噴物件位置辨識。實驗架構採用鋁擠型支架固定噴槍與目標物板子，並設置 LED 輔助光源以降低環境干擾。
在研究方法中，首先對深度攝影機捕捉的影像進行前處理，利用深度學習模型進行語意分割，提取噴槍及目標物的特徵點與位置資訊。選擇的深度學習模型包括 YOLOv8 與 Mask R-CNN。接著應用點雲濾波技術去除點雲數據中的背景雜訊與多餘資訊，從而精確提取場景中目標的三維座標數據。隨後，根據提取的三維數據計算出噴槍與目標物之間的相對距離與角度。此過程結合了語意分割、點雲處理與幾何計算等多項技術，確保數據的準確性與可靠性。研究結果顯示，YOLOv8 在即時處理效能方面表現出色，而 Mask R-CNN 則在精度與細節分割能力上更具優勢。實驗結果進一步顯示，系統對角度與距離的測量誤差均在可接受範圍內，驗證了系統模型的穩定性與準確性。綜合模型的表現，本研究證實結合深度學習與點雲分析技術能有效提升噴漆作業中辨識噴槍姿態的精度與效率，展現出其在工業應用中的潛力。

摘要(英)

This study aims to develop a spray gun pose recognition system for automotive painting, leveraging deep learning and point cloud analysis to address the inefficiencies and inaccuracies associated with traditional manual inspection methods. By integrating a deep learning model and a ZED 2 depth camera, the system facilitates the identification of spray gun pose and the position of target object. The experimental setup employs aluminum extrusion supports to fix the spray gun and target plate while incorporating LED auxiliary lighting to mitigate environmental influence.
In the proposed methodology, images captured by the depth camera undergo preprocessing, followed by semantic segmentation using deep learning models to extract key features and positional information of the spray gun and target object. The selected deep learning models include YOLOv8 and Mask R-CNN. Subsequently, point cloud filtering techniques are applied to remove background noise and extraneous data, enabling precise extraction of the three-dimensional coordinates of the targets. Based on the extracted 3D data, the relative distance and angle between the spray gun and the target object are calculated. This process combines multiple techniques, including semantic segmentation, point cloud processing, and geometric computation, to ensure data accuracy and reliability.
The experimental results indicate that YOLOv8 excels in real-time processing performance, while Mask R-CNN demonstrates superior accuracy and detailed segmentation capabilities. Additionally, the system’s measurements of angles and distances show acceptable error margins, verifying the stability and accuracy of the proposed model. Overall, the findings confirm that integrating deep learning with point cloud analysis can significantly enhance the precision and efficiency of spray gun pose recognition in painting operation, highlighting its potential for industrial applications.

關鍵字(中)

★ 姿態辨識
★ 人工智慧

關鍵字(英)

★ Mask R-CNN
★ YOLOv8

論文目次

摘要 I
致謝 III
表目錄 VI
圖目錄 VII
第一章、前言 1
1.1. 汽車製造廠噴漆作業 1
1.2. 物體姿態辨識 2
1.3. 深度學習應用於影像辨識 4
1.4. 特徵點辨識 8
1.5. 研究目的 10
第二章、研究方法與步驟 12
2.1. 實驗架構與流程 12
2.2. 影像辨識 17
2.2.1. Mask R-CNN 17
2.2.2. YOLOv8 24
2.2.3. 資料集 31
2.3. 特徵點辨識 33
2.3.1. 影像去噪 33
2.3.2. 影像辨識 35
2.3.3. 輪廓偵測 37
2.3.4. 特徵點偵測 38
2.4. 姿態辨識 41
2.4.1. 點雲濾波 41
2.4.2. 點雲數據提取 47
2.4.3. 距離與角度計算 51
第三章、結果與討論 54
3.1. 影像處理結果 54
3.1.1. 模型評估 54
3.1.2. 不同模型的辨識結果比較 60
3.2. 姿態辨識結果 62
第四章、結論 72
第五章、未來研究方向 73
參考文獻 74

參考文獻

1. M. N. Popa and E. Balan, “Particularities of the Painting Process in the Automotive Industry,” Conference Proceedings of the Academy of Romanian Scientists, Vol. 12, pp. 115-128, 2020.
2. N. K. Akafuah, S. Poozesh, A. Salaimeh, G. Patrick, K. Lawler, and K. Saito, “Evolution of the Automotive Body Coating Process—A Review,” Coatings, Vol. 2, 6020024, 2016.
3. B. Zhang, J. Wu, L. Wang, and Z. Yu, “Accurate Dynamic Modeling and Control Parameters Design of an Industrial Hybrid Spray-Painting Robot,” Robotics and Computer Integrated Manufacturing, Vol. 63, 101923, 2020.
4. 國瑞汽車股份有限公司提供。
5. M. Franzo, A. Pica, S. Pascucci, F. Marinozzi, and F. Bini, “Hybrid System Mixed Reality and Marker-Less Motion Tracking for Sports Rehabilitation of Martial Arts Athletes,” Applied Sciences, Vol. 13, pp. 2587-2600, 2023.
6. J. Yu, K. Weng, G. Liang, and G. Xie, “A Vision-Based Robotic Grasping System Using Deep Learning for 3D Object Recognition and Pose Estimation,” 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1175-1180, Shenzhen, China, 2013.
7. M. Seker, A. Mannisto, A. Iosifidis, and J. Raitoharju, “Automatic Social Distance Estimation from Images: Performance Evaluation, Test Benchmark, and Algorithm,” Machine Learning with Applications, Vol. 10, 100427, 2021.
8. W. Yan, Z. Xu, X. Zhou, Q. Su, S. Li, and H. Wu, “Fast Object Pose Estimation Using Adaptive Threshold for Bin-Picking,” IEEE Access, Vol. 8, pp. 63055-63064, 2020.
9. J. Wang, Z. Gao, Y. Zhang, J. Zhou, J. Wu, and P. Li, “Real-Time Detection and Location of Potted Flowers Based on a ZED Camera and a YOLO V4-Tiny Deep Learning Algorithm,” Horticulturae, Vol. 8, 21, 2022.
10. X. Gao, Q. Lizhe, M. Chuangjia, and Y. Sun, “Research on Real-Time Cloth Edge Extraction Method Based on ENet Semantic Segmentation,” Journal of Engineered Fibers and Fabrics, Vol. 17, pp. 1-11, 2022.
11. Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3D ShapeNets: A Deep Representation for Volumetric Shapes,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1912-1920, Boston, MA, USA, 2015.
12. A. Mousavian, D. Anguelov, J. Flynn, and J. Kosecka, “3D Bounding Box Estimation Using Deep Learning and Geometry,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7074-7082, Honolulu, HI, USA, 2017.
13. C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on Point Sets for 3D Classification and Segmentation,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85, Honolulu, HI, USA, 2017.
14. C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space,” Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 5105-5114, Long Beach, CA, USA, 2017.
15. Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, “PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes,” CoRR, Vol. 1711, 00199, 2017.
16. M. Rad and V. Lepetit, “BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting The 3D Poses of Challenging Objects Without Using Depth,” 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3848-3856, Venice, Italy, 2017.
17. W. Kehl, F. Manhardt, F. Tombari, S. Ilic, and N. Navab, “SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again,” 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1530-1538, Venice, Italy, 2017.
18. Y. Li, G. Wang, X. Ji, Y. Xiang, and D. Fox, “DeepIM: Deep Iterative Matching for 6D Pose Estimation,” International Journal of Computer Vision, Vol. 128, pp. 657-678, 2020.
19. C. Wang, D. Xu, Y. Zhu, M.-M. Roberto, C. Lu, F.-F. Li, and S. Savarese, “Densefusion: 6D Object Pose Estimation by Iterative Dense Fusion,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3338-3347, Long Beach, CA, USA, 2019.
20. M. Fu and W. Zhou, “Deephmap++: Combined Projection Grouping and Correspondence Learning for Full Dof Pose Estimation,” Sensors, Vol. 19, pp. 1032-1050, 2019.
21. Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object Detection in 20 Years: A Survey,” Proceedings of the IEEE, Vol. 111, pp. 257-276, 2023.
22. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-Based Convolutional Networks for Accurate Object Detection and Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, pp. 142-158, 2016.
23. S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image Segmentation Using Deep Learning: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, pp. 3523-3542, 2022.
24. A. Kirillov, K. He, R. Girshick, C. Rother, and P Dollar, “Panoptic Segmentation,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9396-9405, Long Beach, CA, USA, 2018.
25. Y. Li, S. Wang, Q. Tian, and X. Ding, “A Survey of Recent Advances in Visual Feature Detection,” Neurocomputing, Vol. 149, pp. 736-751, 2015.
26. W. Yin, H. Wen, Z. Ning, J. Ye, Z. Dong, and L. Luo, “Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks,” Frontiers in Robotics and AI, Vol. 8, 626989, 2021.
27. Medium, [物件偵測] S9: Mask R-CNN 簡介, https://ivan-eng-murmur.medium.com/物件偵測-S9-Mask-R-CNN-簡介-99370c98de28, accessed on November 25, 2024.
28. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, Las Vegas, NV, USA, 2016.
29. S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, ”Aggregated Residual Transformations for Deep Neural Networks,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1492-1500, Honolulu, HI, USA, 2017.
30. T. Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, ”Feature Pyramid Networks for Object Detection,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117-2125, Honolulu, HI, USA, 2017.
31. iT 邦幫忙,【Day 8】物件偵測的標準 IoU (Intersection over Union), https://ithelp.ithome.com.tw/m/articles/10350081, accessed on November 25, 2024.
32. K. He, G. Gkioxari, and P. Dollar, “Mask R-CNN,” Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2961-2969, Venice, Italy, 2017.
33. J. Long, E. Shelhamer, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, Boston, MA, USA, 2015.
34. CSDN, MaskRCNN源碼解析5：損失部分解析, https://blog.csdn.net/sxlsxl119/article/details/103433078, accessed on November 25, 2024.
35. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, pp. 1137-1149, 2017.
36. R. Girshick, “Fast R-CNN,” IEEE International Conference on Computer Vision (ICCV), pp. 1440-1448, Santiago, Chile, 2015.
37. Medium, [物件偵測] S2: Fast R-CNN 簡介, https://ivan-eng-murmur.medium.com/obeject-detection-s2-fast-rcnn-簡介-40cfe7b5f605, accessed on November 25, 2024.
38. Stack overflow, What is the Loss Function of the Mask RCNN?, https://stackoverflow.com/questions/46272841/what-is-the-loss-function-of-the-mask-rcnn, accessed on November 25, 2024.
39. R. Bai, M. Wang, Z. Zhang, J. Lu, and F. Shen, “Automated Construction Site Monitoring Based on Improved YOLOv8-seg Instance Segmentation Algorithm,” IEEE Access, Vol. 11, pp. 139082-139096, 2023.
40. CSDN, YOLOv8 is Here | Detailed Interpretation of the Improved Modules in YOLOv8! YOLOv5 Officially Releases YOLOv8!, https://blog.csdn.net/qq_40716944/article/details/128609569, accessed on December 6, 2024.
41. Medium, [Object detection] YOLOv8詳解, https://henry870603.medium.com/object-detection-yolov8詳解-fdf8874e5e99, accessed on December 6, 2024.
42. CSDN, 萬字詳解 YOLOv8 網路結構：Backbone、Neck、Head，以及 Conv、Bottleneck、C2f、SPPF、Detect 等模組, https://blog.csdn.net/shangyanaf/article/details/139223155, accessed on December 6, 2024.
43. CSDN, YOLOv8 優化：卷積變體——分布移位卷積（DSConv），提升卷積層的內存效率與速度, https://blog.csdn.net/m0_63774211/article/details/130408988, accessed on December 6, 2024.
44. M. G. D. Nascimento, R. Fawcett, and V. A. Prisacariu, “DSConv: Efficient Convolution Operator,” Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5148-5157, Seoul, Korea, 2019.
45. S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, “Path Aggregation Network for Instance Segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8759-8768, Salt Lake City, UT, USA, 2018.
46. CSDN, YOLOv8 的 C2f 模組—與 YOLOv5 的 C3 模組對比, https://blog.csdn.net/python_plus/article/details/129223831, accessed on December 6, 2024.
47. M. Yaseen, “What is YOLOv8: An In-Depth Exploration of the Internal Features of the Next-Generation Object Detector,” 10.48550/arXiv.2408.15857, 2024.
48. J. Terven, D.-M. Cordova-Esparza, and J.-A. Romero-Gonzalez, “A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS,” Machine Learning and Knowledge Extraction, Vol. 5, pp. 1680-1716, 2023.
49. Github, Loss Function Explanation, https://github.com/ultralytics/ultralytics/issues/10465, accessed on December 6, 2024.
50. H. Zhang, Y. Wang, F. Dayoub, and N. Sunderhauf, “VarifocalNet: An IoU-aware Dense Object Detector,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, pp. 8514-8523, 2021.
51. X. Li, W. Wang, L. Wu, S. Chen, X. Hu, J. Li, and J. Yang, “Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection,” Advances in Neural Information Processing Systems, Vol. 33, pp. 21002-21012, 2020.
52. Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression,” Proceedings of the AAAI Conference on Artificial Intelligence,” pp. 12993-13000, Washington, DC, USA, 2020.
53. F. Milletari, N. Navab, and S. A. Ahmadi, “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation,” 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, pp. 565-571, 2016.
54. Roboflow, Roboflow, https://roboflow.com, accessed on December 11, 2024.
55. CSDN, OpenCV筆記：去噪函數—cv2.fastNlMeansDenoising, https://blog.csdn.net/qq_38410428/article/details/93046099, accessed on December 11, 2024.
56. CSDN, Python 去噪函數：OpenCV Python 圖像去噪, https://blog.csdn.net/weixin_40001309/article/details/110326001, accessed on December 11, 2024.
57. CSDN, OpenCV 學習—cv2.findContours() 函數講解 (Python), https://blog.csdn.net/weixin_44690935/article/details/109008946, accessed on December 11, 2024.
58. CSDN, Python + OpenCV 利用函數 cv2.findContours() 和 cv2.drawContours 查找並繪製輪廓, https://blog.csdn.net/weixin_42216109/article/details/89840323, accessed on December 11, 2024.
59. StereoLabs, Depth Settings, https://www.stereolabs.com/docs/depth-sensing/depth-settings, accessed on December 12, 2024.
60. OpenCV, Camera Calibration and 3D Reconstruction, https://docs.opencv.org/4.5.5/d9/d0c/group__calib3d.html, accessed on January 14, 2025.
61. StereoLabs, How can I convert 3D world coordinates to 2D image coordinates and viceversa?, https://support.stereolabs.com/hc/en-us/articles/4554115218711-How-can-I-convert-3D-world-coordinates-to-2D-image-coordinates-and-viceversa, accessed on January 14, 2025.
62. Meshlogic, Fitting a Circle to Cluster of 3D Points, https://meshlogic.github.io/posts/jupyter/curve-fitting/fitting-a-circle-to-cluster-of-3d-points/, accessed on January 14, 2025.

指導教授

林志光(Chih-Kuang Lin)

審核日期

2025-1-17

推文