具即時聽覺辨識之機械手臂應用於螺栓鬆脫檢測之研究

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：102

、訪客IP：3.141.29.202

姓名

蔡沛清(Pei-Ching Tsai) 查詢紙本館藏

畢業系所

土木工程學系

論文名稱

具即時聽覺辨識之機械手臂應用於螺栓鬆脫檢測之研究

相關論文

★ 應用智慧標籤及數據驅動方法於水接觸結構物之結構評估	★ 基於低功耗嵌入式系統及高精度MEMS感測器的智慧鋼索監測系統研發
★ Sensor Code-based Smart Tag Embedded in Concrete for Seepage Sensing Caused by Cracks	★ 智慧型居家機器人用於地震後自動巡查及應變處置之研究
★ 利用UAV整合LoRa與磁導喚醒技術的物聯網架構研發	★ 基於磁吸附與全向輪技術的鋼結構攀爬機器人開發與驗證
★ 基於微型機器學習的智能避障系統在外牆檢測自主移動機器人中的應用	★ 基於ROS的遠端自動多螺栓檢測機器人系統開發
★ 基於BERT語意分析模型的智慧型BIM資訊搜尋問答系統之研究	★ 基於BIM與無線喚醒物聯網裝置之智慧化結構檢測系統開發
★ 利用微型機器學習與微控制器即時檢測室內地磚空心缺陷	★ 結合智慧感測標籤與支持向量機快速判定混凝土裂縫位置
★ 應用於鋼結構檢測之高機動型蚇蠖攀爬機器人設計分析及實作驗證	★ 混凝土缺陷自動修補機器人之研發
★ 研發具邊緣運算能力之無線振動量測裝置應用於橋梁鋼索特徵頻率偵測	★ 結合智慧感測標籤與機器學習方法判別混凝土內部鋼筋鏽蝕可能性之研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-7-24以後開放)

摘要(中)

目前螺栓連接的檢測方式目前在實務上多為檢測人員以橡膠槌敲擊螺栓，聽由敲擊螺栓後所產生的聲音來判斷螺栓是否鬆脫，然而，只依靠人力來進行檢測相當費時，判斷鬆脫需要人員主觀判斷，檢測場所人員不易進入甚至危險。若是能夠透過機械裝置對螺栓進行敲擊與辨識，可以減少檢測所需的人力與時間成本，增加檢測人員進行檢測時的安全，也能透過客觀的方式進行檢測，增加檢測準確率。因此，本研究運用微型機器學習的理念，將梅爾頻率倒譜係數與卷積神經網路部署至微控制器中，研發一套基於機械手臂與即時聽覺辨識功能之螺栓鬆脫檢測系統，用於檢測螺栓是否鬆脫。螺栓的種類被分為鬆脫與緊固兩類，將敲擊螺栓後產生的音訊透過梅爾頻率倒譜係數進行特徵提取，獲得特徵圖，再將音訊的特徵圖輸入至卷積神經網路進行訓練。將訓練完成後的模型與梅爾頻率倒譜系數組成聲學辨識模型並部署至具有麥克風模組的微控制器組成AI聲學辨識模組，結合機械手臂與微型敲擊裝置與主控制器並進行實驗驗證。本研究所訓練之聲學辨識模型在驗證集準確率為100%，在測試集準確率為99.57%。將聲學辨識模型部署至微控制器組成AI聲學辨識模組進行實驗驗證，在低環境噪音時準確率為75.7%，召回率為77.3%，在高環境噪音時準確率為72%，召回率為68.7%。未來透過本系統，可以使人員不需要親自進行敲擊，只需要遠端操作機械手臂就能夠進行敲擊與辨識，降低人員在進行檢測的風險，也能夠客觀且可以快速判斷螺栓是否鬆脫。

摘要(英)

Currently, the detection of bolt connections in practical applications often involves inspectors using rubber hammers to percuss the bolts and check if they are loosened based on the generated sound. However, this manual detection process is time-consuming, requires subjective judgment by the inspectors, and may be difficult or even dangerous to access certain detection sites.
To reduce the manpower and time costs of inspections, enhance the safety of inspectors, and improve the accuracy of detection through an objective approach, this study applies the concept of Tiny Machine Learning (TinyML) by deploying Mel-frequency cepstral coefficients (MFCCs) and Convolutional Neural Networks (CNNs) to a microcontroller. A bolt-loosen detection system based on a robotic arm and real-time audio recognition capabilities is developed for detecting loosen bolts. The bolts are categorized into two types: loosen and tight. The audio signals generated by knocking the bolts are subjected to feature extraction using MFCCs to obtain feature maps, which are then input to the CNN for training. After training, the model is combined with MFCCs to form an acoustic recognition model. The acoustic recognition model is then deployed to a microcontroller equipped with a microphone module, creating an AI acoustic recognition module. The AI acoustic recognition module is combined with a robotic arm and a tiny knocking device and a main controller for experimental verification.
The trained acoustic recognition model achieved an accuracy of 100% on the validation set and 99.57% on the test set. The AI acoustic recognition module deployed to the microcontroller achieved an accuracy of 75.7% and a recall rate of 77.3% in low environmental noise, and an accuracy of 72% and a recall rate of 68.7% in high environmental noise.
In the future, using this system, personnel will not need to perform the percussing manually; instead, they can remotely operate the mechanical arm for tapping and recognition, reducing the risks associated with inspections and providing an objective and fast way to determine whether the bolts are loosened.

關鍵字(中)

★ 螺栓鬆脫檢測
★ 敲擊法
★ 微型機器學習
★ 梅爾頻率倒譜係數
★ 卷積神經網路

關鍵字(英)

★ Bolt Loosening Detection
★ Percussion Method
★ Tiny Machine Learning
★ Mel-Frequency Cepstral Coefficients
★ Convolutional Neural Network

論文目次

摘要 i
Abstract ii
致謝 iv
目錄 v
圖目錄 vii
表目錄 ix
一、緒論 1
1-1 研究動機 1
1-2 研究目的 2
1-3 論文架構 3
二、文獻回顧 4
2-1 螺栓鬆脫檢測技術 4
2-2 微型機器學習 6
2-3 卷積神經網路 7
2-4 梅爾頻率倒譜係數 8
三、研究方法 9
3-1 基於機械手臂與即時聽覺辨識功能之螺栓鬆脫檢測系統架構 9
3-2 螺栓敲擊音訊訊號處理與分析 11
3-3 聲學辨識模型架構與訓練 28
3-4 混淆矩陣 34
3-5 基於機械手臂與即時聽覺辨識功能之螺栓鬆脫檢測系統之演算法 36
四、實驗規劃與設計 39
4-1 聲學辨識模型訓練資料收集實驗規劃 39
4-2 基於機械手臂與即時聽覺辨識功能之螺栓鬆脫檢測系統實驗驗證規劃 41
五、實驗結果與討論 44
5-1 聲學辨識模型訓練探討 44
5-2 基於機械手臂與即時聽覺辨識功能之螺栓鬆脫檢測系統實驗驗證 49
5-3 螺栓鬆脫檢測相關文獻比較與探討 52
5-4 本研究之限制與未來改善方向 55
六、結論與未來展望 56
6-1 結論 56
6-2 未來展望 56
參考文獻 57

參考文獻

1. Zhang, Y., X. Zhao, X. Sun, W. Su, and Z. Xue, Bolt loosening detection based on audio classification. Advances in Structural Engineering, 2019. 22(13): p. 2882-2891. https://doi.org/10.1177/1369433219852565.
2. Yuan, R., Y. Lv, S. Xu, L. Li, Q. Kong, and G. Song, ResNet-integrated very early bolt looseness monitoring based on intrinsic feature extraction of percussion sounds. Smart Materials and Structures, 2023. 32(3): p. 034002. https://doi.org/10.1088/1361-665X/acb2a0.
3. Wang, F., X. Chen, and G. Song. Percussion-based Detection of Bolt Looseness Using Speech Recognition Technology and Least Square Support Vector Machine. in 2020 IEEE International Conference on Networking, Sensing and Control (ICNSC). 2020. IEEE. https://doi.org/10.1109/ICNSC48988.2020.9238108.
4. Wang, F. and G. Song, Bolt-looseness detection by a new percussion-based method using multifractal analysis and gradient boosting decision tree. Structural Health Monitoring, 2020. 19(6): p. 2023-2032. https://doi.org/10.1177/1475921720912780.
5. Wang, F. and G. Song, 1D-TICapsNet: An audio signal processing algorithm for bolt early looseness detection. Structural Health Monitoring, 2020: p. 1475921720976989. https://doi.org/10.1177/1475921720976989.
6. Wang, F., A. Mobiny, H. Van Nguyen, and G. Song, If structure can exclaim: a novel robotic-assisted percussion method for spatial bolt-ball joint looseness detection. Structural Health Monitoring, 2021. 20(4): p. 1597-1608. https://doi.org/10.1177/1475921720923147.
7. Park, J., T. Kim, and J. Kim. Image-based bolt-loosening detection technique of bolt joint in steel bridges. in 6th International Conference on Advances in Experimental Structural Engineering 11th International Workshop on Advanced Smart Materials and Smart Structures Technology. 2015. http://sstl.cee.illinois.edu/papers/aeseancrisst15/219_Park_Image-Based.pdf.
8. Huang, J., J. Liu, H. Gong, and X. Deng, A comprehensive review of loosening detection methods for threaded fasteners. Mechanical Systems and Signal Processing, 2022. 168: p. 108652. https://doi.org/10.1016/j.ymssp.2021.108652.
9. Miao, R., R. Shen, S. Zhang, and S. Xue, A review of bolt tightening force measurement and loosening detection. Sensors, 2020. 20(11): p. 3165. https://doi.org/10.3390/s20113165.
10. Zhang, Y., X. Sun, K.J. Loh, W. Su, Z. Xue, and X. Zhao, Autonomous bolt loosening detection using deep learning. Structural Health Monitoring, 2020. 19(1): p. 105-122. https://doi.org/10.1177/1475921719837509.
11. Kong, X. and J. Li, Image registration-based bolt loosening detection of steel joints. Sensors, 2018. 18(4): p. 1000. https://doi.org/10.3390/s18041000.
12. Huynh, T.-C., J.-H. Park, H.-J. Jung, and J.-T. Kim, Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing. Automation in Construction, 2019. 105: p. 102844. https://doi.org/10.1016/j.autcon.2019.102844.
13. Pham, H.C., Q.-B. Ta, J.-T. Kim, D.-D. Ho, X.-L. Tran, and T.-C. Huynh, Bolt-loosening monitoring framework using an image-based deep learning and graphical model. Sensors, 2020. 20(12): p. 3382. https://doi.org/10.3390/s20123382.
14. Pan, X., S. Tavasoli, and T. Yang, Autonomous 3D vision‐based bolt loosening assessment using micro aerial vehicles. Computer‐Aided Civil and Infrastructure Engineering, 2023. https://doi.org/10.1111/mice.13023.
15. Eraliev, O., K.-H. Lee, and C.-H. Lee, Vibration-based loosening detection of a multi-bolt structure using machine learning algorithms. Sensors, 2022. 22(3): p. 1210. https://doi.org/10.3390/s22031210.
16. Nikravesh, S.M.Y. and M. Goudarzi, A review paper on looseness detection methods in bolted structures. Latin American Journal of Solids and Structures, 2017. 14: p. 2153-2176. https://doi.org/10.1590/1679-78254231.
17. Chen, D., L. Huo, and G. Song, EMI based multi-bolt looseness detection using series/parallel multi-sensing technique. Smart Struct. Syst, 2020. 25: p. 423-432. https://doi.org/10.12989/sss.2020.25.4.423.
18. Nguyen, T.-T., Q.-B. Ta, D.-D. Ho, J.-T. Kim, and T.-C. Huynh, A method for automated bolt-loosening monitoring and assessment using impedance technique and deep learning. Developments in the Built Environment, 2023. 14: p. 100122. https://doi.org/10.1016/j.dibe.2023.100122.
19. Li, X.-X., D. Li, W.-X. Ren, and J.-S. Zhang, Loosening Identification of Multi-Bolt Connections Based on Wavelet Transform and ResNet-50 Convolutional Neural Network. Sensors, 2022. 22(18): p. 6825. https://doi.org/10.3390/s22186825.
20. Ying, T., D. Zhu, Y. Zou, Y. Huang, and P. Zhao. Design of restaurant intelligent seat-seeking system based on ESP32. in 2023 IEEE 2nd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA). 2023. IEEE. https://doi.org/10.1109/EEBDA56825.2023.10090804.
21. Abhinay, D., S.V. Vighnesh, L.K. Durgam, and R.K. Jatoth. Real-time Classification of Vehicle Logos on Arduino Nano BLE using Edge Impulse. in 2023 4th International Conference on Signal Processing and Communication (ICSPC). 2023. IEEE. https://doi.org/10.1109/ICSPC57692.2023.10126068.
22. Altayeb, M., M. Zennaro, and E. Pietrosemoli, TinyML Gamma Radiation Classifier. Nuclear Engineering and Technology, 2022. https://doi.org/10.1016/j.net.2022.09.032.
23. Luukkonen, T., A. Colley, T. Seppänen, and J. Häkkilä. Cough activated dynamic face visor. in Augmented Humans Conference 2021. 2021. https://doi.org/10.1145/3458709.3459000.
24. Ogore, M.M., K. Nkurikiyeyezu, and J. Nsenga. Offline Prediction of Cholera in Rural Communal Tap Waters Using Edge AI inference. in 2021 IEEE Globecom Workshops (GC Wkshps). 2021. IEEE. https://doi.org/10.1109/GCWkshps52748.2021.9682128.
25. Krayden, A., M. Schohet, O. Shmueli, D. Shlenkevitch, T. Blank, S. Stolyarova, and Y. Nemirovsky. CMOS-MEMS Gas Sensor Dubbed GMOS for Selective Analysis of Gases with Tiny Edge Machine Learning. in Presented at the 9th International Electronic Conference on Sensors and Applications. 2022. https://doi.org/10.3390/ecsa-9-13316.
26. Jailani, M.H.A., K.A. Mohamad, A. Alias, and M.S. Nordin, Development of Water Sound Analyzer for An Automatic Fertilizer System in Agriculture Industry. Evolution in Electrical and Electronic Engineering, 2022. 3(2): p. 351-359. https://penerbit.uthm.edu.my/periodicals/index.php/eeee/article/view/8581.
27. Stege, M., C. Orfanidis, and X. Fafoutis. Plantar Biometrics for Edge Computing. in BodySys@ MobiSys. 2022. https://doi.org/10.1145/3539489.3539589.
28. Ooko, S.O., D. Mukanyiligira, J.P. Munyampundu, and J. Nsenga. Edge AI-based respiratory disease recognition from exhaled breath signatures. in 2021 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT). 2021. IEEE. https://doi.org/10.1109/JEEIT53412.2021.9634140.
29. LeCun, Y., K. Kavukcuoglu, and C. Farabet. Convolutional networks and applications in vision. in Proceedings of 2010 IEEE international symposium on circuits and systems. 2010. IEEE. https://doi.org/10.1109/ISCAS.2010.5537907.
30. LeCun, Y., Y. Bengio, and G. Hinton, Deep learning. nature, 2015. 521(7553): p. 436-444. https://doi.org/10.1038/nature14539.
31. Kattenborn, T., J. Leitloff, F. Schiefer, and S. Hinz, Review on Convolutional Neural Networks (CNN) in vegetation remote sensing. ISPRS journal of photogrammetry and remote sensing, 2021. 173: p. 24-49. https://doi.org/10.1016/j.isprsjprs.2020.12.010.
32. Bharadiya, J., Convolutional Neural Networks for Image Classification. International Journal of Innovative Science and Research Technology, 2023. 8(5): p. 673-677. https://doi.org/10.5281/zenodo.7952031.
33. Park, E., CRNet: a multimodal deep convolutional neural network for customer revisit prediction. Journal of big Data, 2023. 10(1): p. 1-10. https://doi.org/10.1186/s40537-022-00674-4.
34. Dörfler, M., R. Bammer, and T. Grill. Inside the spectrogram: Convolutional Neural Networks in audio processing. in 2017 international conference on sampling theory and applications (SampTA). 2017. IEEE. https://doi.org/10.1109/SAMPTA.2017.8024472.
35. Kanke, R.G., R.M. Gaikwad, and M.R. Baheti, Enhanced Marathi Speech Recognition Using Double Delta MFCC and DTW. International Journal of Digital Technologies, 2023. 2(1). https://journal.nielit.edu.in/index.php/01/article/view/33.
36. Yee, C.S. and A.M. Ahmad. Malay language text-independent speaker verification using NN-MLP classifier with MFCC. in 2008 International Conference on Electronic Design. 2008. IEEE. https://doi.org/10.1109/ICED.2008.4786666.
37. Anabeza, C.C., G.B.S. Limt, M.L.P. Velasco, E. Svbingco, and D.D. Ligutan. DTW Threshold Determination for English Word Utterances in Filipino Accent using MFCC. in 2023 17th International Conference on Ubiquitous Information Management and Communication (IMCOM). 2023. IEEE. https://doi.org/10.1109/IMCOM56909.2023.10035607.
38. Mannar Mannan, J., Human Emotion Recognize Using Convolutional Neural Network (CNN) and Mel Frequency Cepstral Coefficient (MFCC). https://seyboldpublications.com/wp-content/uploads/2023/07/manJ41982.pdf.
39. Ramadevi, C., K. Anusha, and P. Thummeti, CNN and MFCC based Speech Net: Children Speech Recognition model. https://jcdronline.org/admin/Uploads/Files/63fc59a040bec0.68653996.pdf.
40. Nived, B.V., K. Jamal, G. Mahesh, and R.M. Kumar. Design of Custom Keyword Recognition using Edge Impulse on Arduino Nano 33 BLE Sense. in 2023 2nd International Conference on Applied Artificial Intelligence and Computing (ICAAIC). 2023. IEEE. https://doi.org/10.1109/ICAAIC56838.2023.10140757.
41. Muda, L., M. Begam, and I. Elamvazuthi, Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. arXiv preprint arXiv:1003.4083, 2010. https://doi.org/10.48550/arXiv.1003.4083.
42. Xu, H., X. Zhang, and L. Jia. The extraction and simulation of Mel frequency cepstrum speech parameters. in 2012 International Conference on Systems and Informatics (ICSAI2012). 2012. IEEE. https://doi.org/10.1109/ICSAI.2012.6223385.

指導教授

林子軒(Tzu-Hsuan Lin)

審核日期

2023-7-25

推文