應用於攀爬檢測機器人之輕量級即時多目標螺栓缺陷影像檢測系統之研究

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：43

、訪客IP：3.141.45.104

姓名

張建達(Chien-Ta Chang) 查詢紙本館藏

畢業系所

土木工程學系

論文名稱

應用於攀爬檢測機器人之輕量級即時多目標螺栓缺陷影像檢測系統之研究
(Research on Lightweight Real-Time Multi-Object Bolt Defect Image Detection System Applied to Climbing Inspection Robots)

相關論文

★ 基於低功耗嵌入式系統及高精度MEMS感測器的智慧鋼索監測系統研發	★ Sensor Code-based Smart Tag Embedded in Concrete for Seepage Sensing Caused by Cracks
★ 智慧型居家機器人用於地震後自動巡查及應變處置之研究	★ 利用UAV整合LoRa與磁導喚醒技術的物聯網架構研發
★ 基於BERT語意分析模型的智慧型BIM資訊搜尋問答系統之研究	★ 基於BIM與無線喚醒物聯網裝置之智慧化結構檢測系統開發
★ 利用微型機器學習與微控制器即時檢測室內地磚空心缺陷	★ 結合智慧感測標籤與支持向量機快速判定混凝土裂縫位置
★ 應用於鋼結構檢測之高機動型蚇蠖攀爬機器人設計分析及實作驗證	★ 混凝土缺陷自動修補機器人之研發
★ 研發具邊緣運算能力之無線振動量測裝置應用於橋梁鋼索特徵頻率偵測	★ 結合智慧感測標籤與機器學習方法判別混凝土內部鋼筋鏽蝕可能性之研究
★ 具即時聽覺辨識之機械手臂應用於螺栓鬆脫檢測之研究	★ 生物啟發式攀爬機器人應用於外牆磁磚缺陷檢測之研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-7-24以後開放)

摘要(中)

在鋼結構工程中，利用螺栓連接鋼材部件是一種十分常見的方法，螺栓可以保證各個結構部件之間穩固相連。然而，隨著氣候、外力、地震等自然或人為因素影響，螺栓可能會因此產生脫落、鬆動等缺陷情形，導致鋼構造物的安全性降低。目前螺栓檢測工作，大多通過檢測人員利用橡膠槌等工具敲擊螺栓以人工判斷其鬆動情形，這些檢測工作繁瑣費力，且可能遇到安全風險或是檢測困難的情形。若能夠以小型儀器對於缺陷情形進行自動化檢測，不僅能夠提高檢測效率，也能夠降低檢修人員進行檢修工作時的困難。因此，本研究結合Tiny Machine Learning的理念以及微控制器，開發可用於攀爬檢測機器人之輕量級即時多目標螺栓缺陷影像檢測系統，用於對螺栓的缺陷樣態進行檢測。螺栓的樣態被分為正常(Normal)、鬆動(Loosen)、欠缺(Miss)三種，利用FOMO (Faster Objects, More Objects)演算法進行視覺檢測任務，並將訓練後的模型部署於視覺檢測模組中，結合磁吸攀爬機器人進行實際螺栓檢測測試。本研究所訓練之模型在驗證集中獲得74.8%的F1分數，並且於測試集獲得72.9%的F1分數，而部署於視覺檢測模組上的量化模型則在驗證集中獲得72.4%的F1分數。除此之外，實際進行室外箱梁檢測試驗，在10個Case中，表現最好的Case所獲得的精確率及召回率分別為89%及82%，最差的則是57%與67%，而所有Case之精確率及召回率平均值皆為77%和76%。通過該缺陷檢測系統之結果，並且能夠及時回傳影像供使用者查看螺栓辨識結果，供檢測人員在遠端就能操控設備執行檢測任務，且整個系統更為輕量化和低能量消耗，並且能夠配置於不同的載具當中進行檢測任務。未來，螺栓缺陷檢測能夠變的更為容易且降低人力需求，並且整體檢測工作所需的儀器成本能夠下降，輕量化和高速檢測更是未來螺栓檢測工作的目標。

摘要(英)

In steel structure engineering, using bolts to connect steel components is a common method that ensures a secure connection between structural elements. However, due to natural or human factors such as climate, external forces, and earthquakes, bolts may experience defects such as loosening or detachment, leading to a decrease in the safety of steel structures. Currently, bolt inspection work is mostly performed by inspectors manually checking for loosening by using rubber hammers and other tools. These inspection tasks are labor-intensive, time-consuming, and may involve safety risks or difficulties in inspection.

To address this, this study combines the concept of Tiny Machine Learning with microcontrollers to develop a lightweight real-time multi-target bolt defect image detection system for climbing inspection robots. This system is used to detect the defect patterns of bolts, which are categorized into "Normal," "Loosen," and "Miss." The Faster Objects, More Objects (FOMO) algorithm is used for the visual detection task, and the trained model is deployed in the visual detection module, combined with a magnetic climbing robot for actual bolt inspection tests.

The trained model achieved an F1 score of 74.8% on the validation set and an F1 score of 72.9% on the test set, while the quantized model deployed on the microcontroller achieved an F1 score of 72.4% on the validation set. In actual outdoor box beam tests, the best-case achieved precision and recall rates of 89% and 82%, respectively, while the worst-case achieved 57% and 67%. The average precision and recall rates for all cases were 77% and 76%, respectively.

The defect detection system can provide real-time feedback on the bolt recognition results, allowing inspectors to remotely control the device for inspection tasks. The entire system is lightweight, low-energy consuming, and can be configured on different carriers for inspection tasks. In the future, bolt defect detection is expected to become easier and reduce manpower requirements. The overall instrument cost for inspection work can be reduced, and the goals of lightweight and high-speed detection are pursued in future bolt inspection tasks.

關鍵字(中)

★ 螺栓缺陷檢測
★ 微型機器學習
★ 卷積神經網路
★ 目標檢測
★ 視覺辨識

關鍵字(英)

★ Bolt defect detection
★ Tiny machine learning
★ Convolutional neural networks
★ Object detection
★ Visual recognition

論文目次

摘要 i
Abstract ii
致謝 iv
目錄 v
圖目錄 vii
表目錄 x
一、緒論 1
1-1 研究動機 1
1-2 研究目的 2
1-3 論文架構 3
二、文獻回顧 4
2-1 螺栓鬆脫成因 4
2-2 現行螺栓鬆脫檢測技術及研究 4
2-3 微型機器學習與視覺檢測技術發展 6
2-4 過去文獻研究總結 7
三、研究方法 8
3-1 用於攀爬檢測機器人之輕量級即時多目標螺栓缺陷影像檢測系統 8
3-2 視覺檢測演算法之架構 12
3-2-1 FOMO模型架構 13
3-2-2 深度可分離卷積 18
3-2-3 寬度係數 21
3-2-4 反向瓶頸殘差模組 21
3-3 即時多目標螺栓缺陷影像檢測系統之演算法 23
四、實驗規劃與設計 25
4-1 用於訓練之螺栓缺陷影像資料及處理 25
4-2 室外箱梁實際檢測實驗規劃 33
五、結果與討論 37
5-1 FOMO模型訓練探討 37
5-2 攀爬機器人實際檢測結果探討 44
Case 1 44
Case 2 46
Case 3 47
Case 4 49
Case 5 50
Case 6 52
Case 7 53
Case 8 55
Case 9 56
Case 10 58
綜合討論 59
5-3 螺栓鬆脫視覺檢測方式比較與探討 60
5-4 本研究之限制以及未來改善方向 61
六、結論與未來展望 63
6-1 結論 63
6-2 未來展望 63
參考文獻 64

參考文獻

[1] Kakirde, A. and Dravid, S., Study of Vibration Loosening of Bolted Joints-A Review. International Journal for Advance Research in Applied Science and Engineering, 2017, 6(Special): p. 988-997. https://www.iaetsdjaras.org/gallery/6-january-399.pdf.
[2] Wang, T., Song, G., Liu, S., Li, Y., and Xiao, H., Review of Bolted Connection Monitoring. International Journal of Distributed Sensor Networks, 2013, 9(12): p. 871213. https://doi.org/10.1155/2013/871213.
[3] Liu, X., Mi, X., Liu, J., Long, L., Cai, Z., Mo, J., Peng, J., and Zhu, M., Axial load distribution and self-loosening behavior of bolted joints subjected to torsional excitation. Engineering Failure Analysis, 2021, 119: p. 104985. https://doi.org/10.1016/j.engfailanal.2020.104985.
[4] Yokoyama, T., Olsson, M., Izumi, S., and Sakai, S., Investigation into the self-loosening behavior of bolted joint subjected to rotational loading. Engineering Failure Analysis, 2012, 23: p. 35-43. https://doi.org/10.1016/j.engfailanal.2012.01.010.
[5] Zhang, Y., Zhao, X., Sun, X., Su, W., and Xue, Z., Bolt loosening detection based on audio classification. Advances in Structural Engineering, 2019, 22(13): p. 2882-2891. https://doi.org/10.1177/1369433219852565.
[6] Yuan, C., Wang, S., Qi, Y., and Kong, Q., Automated structural bolt looseness detection using deep learning-based prediction model. Structural Control and Health Monitoring, 2022, 29(3): p. e2899. https://doi.org/10.1002/stc.2899.
[7] Wang, F. and Song, G., 1D-TICapsNet: An audio signal processing algorithm for bolt early looseness detection. Structural Health Monitoring, 2021, 20(5): p. 2828-2839. https://doi.org/10.1177/1475921720976989.
[8] Wang, F., Ho, S.C.M., and Song, G., Modeling and analysis of an impact-acoustic method for bolt looseness identification. Mechanical Systems and Signal Processing, 2019, 133: p. 106249. https://doi.org/10.1016/j.ymssp.2019.106249.
[9] Wu, J., Cui, X., and Xu, Y., A Novel RFID-Based Sensing Method for Low-Cost Bolt Loosening Monitoring. Sensors, 2016, 16(2): p. 168. https://doi.org/10.3390/s16020168.
[10] Shao, J., Wang, T., Yin, H., Yang, D., and Li, Y., Bolt Looseness Detection Based on Piezoelectric Impedance Frequency Shift. Applied Sciences, 2016, 6(10): p. 298. https://doi.org/10.3390/app6100298.
[11] Wang, F., Chen, Z., and Song, G., Smart crawfish: A concept of underwater multi-bolt looseness identification using entropy-enhanced active sensing and ensemble learning. Mechanical Systems and Signal Processing, 2021, 149: p. 107186. https://doi.org/https://doi.org/10.1016/j.ymssp.2020.107186.
[12] Xing, Y. and Geng, H., Loose bolt detection method based on wavelet packet decomposition with CNN. Journal of Physics: Conference Series, 2023, 2419(1): p. 012038. https://doi.org/10.1088/1742-6596/2419/1/012038.
[13] Park, J.H., Kim, T.H., and Kim, J.T., Image-based Bolt-loosening Detection Technique of Bolt Joint in Steel Bridges. 6th International Conference on Advances in Experimental Structural Engineering 11th International Workshop on Advanced Smart Materials and Smart Structures Technology, 2015: p. 1-2. http://sstl.cee.illinois.edu/papers/aeseancrisst15/219_Park_Image-Based.pdf.
[14] Nguyen, T.-C., Huynh, T.-C., Ryu, J.-Y., Park, J.-H., and Kim, J.-T. Bolt-loosening identification of bolt connections by vision image-based technique. in Nondestructive Characterization and Monitoring of Advanced Materials, Aerospace, and Civil Infrastructure 2016. 2016. p. 227-243. https://doi.org/10.1117/12.2219055.
[15] Kong, X. and Li, J., Image Registration-Based Bolt Loosening Detection of Steel Joints. Sensors, 2018, 18(4): p. 1000. https://doi.org/10.3390/s18041000.
[16] Zhang, Y., Sun, X., Loh, K.J., Su, W., Xue, Z., and Zhao, X., Autonomous bolt loosening detection using deep learning. Structural Health Monitoring, 2020, 19(1): p. 105-122. https://doi.org/10.1177/1475921719837509.
[17] Sun, Y., Li, M., Dong, R., Chen, W., and Jiang, D., Vision-Based Detection of Bolt Loosening Using YOLOv5. Sensors, 2022, 22(14): p. 5184. https://doi.org/10.3390/s22145184.
[18] Pan, X. and Yang, T.Y., Image-based monitoring of bolt loosening through deep-learning-based integrated detection and tracking. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(10): p. 1207-1222. https://doi.org/10.1111/mice.12797.
[19] Jiang, S., Zhang, J., Wang, W., and Wang, Y., Automatic Inspection of Bridge Bolts Using Unmanned Aerial Vision and Adaptive Scale Unification-Based Deep Learning. Remote Sensing, 2023, 15(2): p. 328. https://doi.org/10.3390/rs15020328.
[20] Lao, W., Cui, C., Zhang, D., Zhang, Q., and Bao, Y., Computer Vision-Based Autonomous Method for Quantitative Detection of Loose Bolts in Bolted Connections of Steel Structures. Structural Control and Health Monitoring, 2023, 2023: p. 8817058. https://doi.org/10.1155/2023/8817058.
[21] Lu, Q., Jing, Y., and Zhao, X., Bolt Loosening Detection Using Key-Point Detection Enhanced by Synthetic Datasets. Applied Sciences, 2023, 13(3): p. 2020. https://doi.org/10.3390/app13032020.
[22] Pan, X., Tavasoli, S., and Yang, T.Y., Autonomous 3D vision-based bolt loosening assessment using micro aerial vehicles. Computer-Aided Civil and Infrastructure Engineering, 2023. https://doi.org/10.1111/mice.13023.
[23] Sanchez-Iborra, R. and Skarmeta, A.F., TinyML-Enabled Frugal Smart Objects: Challenges and Opportunities. IEEE Circuits and Systems Magazine, 2020, 20(3): p. 4-18. https://doi.org/10.1109/MCAS.2020.3005467.
[24] LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., and Jackel, L.D., Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1989, 1(4): p. 541-551. https://doi.org/10.1162/neco.1989.1.4.541.
[25] Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P., Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): p. 2278-2324. https://doi.org/10.1109/5.726791.
[26] Krizhevsky, A., Sutskever, I., and Hinton, G.E., Imagenet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6): p. 84-90. https://doi.org/10.1145/3065386.
[27] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. p. 1-9. https://doi.org/10.48550/arXiv.1409.4842.
[28] Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. in International conference on machine learning. pmlr, 2015. p. 448-456. https://doi.org/10.48550/arXiv.1502.03167.
[29] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 2818-2826. https://doi.org/10.48550/arXiv.1512.00567.
[30] Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. Inception-v4, inception-resnet and the impact of residual connections on learning. in Proceedings of the AAAI conference on artificial intelligence. 2017. p. https://doi.org/10.1609/aaai.v31i1.11231.
[31] Simonyan, K. and Zisserman, A., Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. https://doi.org/10.48550/arXiv.1409.1556.
[32] He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 770-778. https://doi.org/10.48550/arXiv.1512.03385.
[33] Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K., SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360, 2016. https://doi.org/10.48550/arXiv.1602.07360.
[34] Chollet, F. Xception: Deep learning with depthwise separable convolutions. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 1251-1258. https://doi.org/10.48550/arXiv.1610.02357.
[35] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H., Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv: 1704.04861, 2017. https://doi.org/10.48550/arXiv.1704.04861.
[36] Zhang, X., Zhou, X., Lin, M., and Sun, J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. arXiv:1707.01083. 2017. https://doi.org/10.48550/arXiv.1707.01083.
[37] Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. arXiv:1506.02640. 2015. https://doi.org/10.48550/arXiv.1506.02640.
[38] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. SSD: Single Shot MultiBox Detector. arXiv:1512.02325. 2015. https://doi.org/10.48550/arXiv.1512.02325.
[39] Edge Impulse, FOMO: Object detection for constrained devices. 2022. https://docs.edgeimpulse.com/docs/edge-impulse-studio/learning-blocks/object-detection/fomo-object-detection-for-constrained-devices.
[40] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv:1801.04381. 2018. https://doi.org/10.48550/arXiv.1801.04381.
[41] Wada, K., Labelme: Image Polygonal Annotation with Python (computer software). https://doi.org/10.5281/zenodo.5711226.
[42] Kingma, D.P. and Ba, J. Adam: A Method for Stochastic Optimization. arXiv:1412.6980. 2014. https://doi.org/10.48550/arXiv.1412.6980.

指導教授

林子軒(Tzu-Hsuan Lin)

審核日期

2023-7-25

推文