無人機於大型物件之自動外觀巡檢

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：38

、訪客IP：3.149.247.106

姓名

戴祥印(Xiang-Yin Dai) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

無人機於大型物件之自動外觀巡檢

相關論文

★ 直接甲醇燃料電池混合供電系統之控制研究	★ 利用折射率檢測法在水耕植物之水質檢測研究
★ DSP主控之模型車自動導控系統	★ 旋轉式倒單擺動作控制之再設計
★ 高速公路上下匝道燈號之模糊控制決策	★ 模糊集合之模糊度探討
★ 雙質量彈簧連結系統運動控制性能之再改良	★ 桌上曲棍球之影像視覺系統
★ 桌上曲棍球之機器人攻防控制	★ 模型直昇機姿態控制
★ 模糊控制系統的穩定性分析及設計	★ 門禁監控即時辨識系統
★ 桌上曲棍球：人與機械手對打	★ 麻將牌辨識系統
★ 相關誤差神經網路之應用於輻射量測植被和土壤含水量	★ 三節式機器人之站立控制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

一般大型物件，如客機、橋梁等之瑕疵檢測，因物件巨大又高，人類檢測必須依靠不同工具或爬高才能達成檢測目的。爬高危險性增加，靠其他工具費時，成本又高，效率也無法提升，因此本論文旨在建立一套基於無人機之大型物件外觀巡檢系統，能使無人機沿著物件外型以最短路徑自動飛行，並且控制雲台自動調整角度，以確保雲台相機的拍攝範圍能完整覆蓋待檢測物件，蒐集到物件的所有外觀影像後，再利用深度學習技術偵測物件外觀是否有瑕疵，協助人類更方便更有效率的完成檢測任務。
考量到室內與室外的通用性，此系統基於視覺之同步定位與地圖構建(Visual Simultaneous Localization and Mapping, V-SLAM)達成無人機的定位，因此不論GPS是否收訊良好皆能進行自動巡檢飛行。此系統可分為三個功能模組，第一部分為面對陌生的待檢測物件時，先透過手動控制無人機飛行，蒐集待檢測物件的360^°影像，便能進行運動恢復結構 (Structure from Motion, SfM) 建立物件的點雲模型，再設計點雲前處理演算法校正點雲模型的誤差。第二部分為透過點雲處理找出適合無人機自動飛行的導航點，以及在各個導航點中找出合適的無人機偏航(yaw)角度與雲台角度，再利用最佳化方法解決旅行推銷員問題(Travelling Salesman Problem, TSP) 找出路徑距離最短的導航點飛行順序，完成導航路徑規劃後，往後面對相同的物件即可直接進行自動巡檢飛行，不須再手動飛行建立點雲。第三部分為外觀瑕疵偵測，系統可偵測出物件外觀影像中的瑕疵，並且根據較常見的瑕疵種類進行分類，本論文考慮的瑕疵是以大型客機的瑕疵為對象，分別有掉漆、鏽蝕、凹痕三種類別。本論文使用OpenSfM建立物件點雲；利用隨機抽樣一致性演算法 (Random Sample Consensus, RANSAC) 與主成分分析 (Principal Component Analysis, PCA) 進行點雲前處理之校正；採用了基因演算法(Genetic Algorithm)計算導航點的飛行順序；使用Scaled-YOLOv4網路架構偵測與辨識物件外觀影像中的瑕疵。本論文在模擬環境與實體實驗中均驗證其可行性，在以汽車為檢測對象的實驗測試中，此系統的拍攝覆蓋率達100%，瑕疵偵測準確率達92.7%，成果顯示本論文提出的方法能拍攝到完整的物件表面影像且偵測出瑕疵。

摘要(英)

In general, the defect inspection of large objects, such as airliners, bridges, etc., must rely on different tools or climb to high places to achieve the inspection because the objects are huge and high. However, climbing to high places is dangerous, and relying on other tools is time-consuming and costly. Therefore, this thesis aims to establish a system for detecting defects in the surface of large objects based on a drone. The system enables the drone to fly along the object′s exterior with the shortest path and adjust the angle of the gimbal such that the drone′s flying ensures complete coverage of the object by the gimbal camera. After the drone collects all the images of the object′s surface, it uses deep learning technology to detect any defects in the object′s surface. The system can help humans to complete the inspection task more easily and efficiently. To consider the usability of indoor and outdoor, the system uses Visual Simultaneous Localization and Mapping (V-SLAM) as the position control for the drone. The drone can fly stably regardless of the signal quality of the GPS. There are three function modules in the system. The first part is to collect 360 degree images of the object by manually controlling the drone when facing an unfamiliar object. Then, we use Structure from Motion (SfM) to build a point cloud of the object. Finally, the point cloud pre-processing algorithm is used to reduce the error of the point cloud. The second part is to find the appropriate navigation points for the drone to fly automatically and look for the suitable yaw angle and gimbal angle at each navigation point. Then we use the optimization method to solve the Travelling Salesman Problem (TSP) to find the shortest fly path. After completing the path planning, the same object can be flown directly for automatic inspection without needing to fly to create a point cloud manually. The third part is the surface defect detection. The system detects defects in the surface image of an object and classifies them according to the more common types of defects. The defects considered in this thesis are paint loss, corrosion, and dent on large airliners. Here, we use the OpenSfM to create point clouds of objects, use Random Sample Consensus (RANSAC), and Principal Component Analysis (PCA) in pre-processing to calibrate the point clouds. Then we use the genetic algorithm to compute flight sequences of navigation points and use the Scaled-YOLOv4 network architecture to detect and classify defects on the surface of objects. The feasibility of this thesis is demonstrated in both simulated and physical experiments. In an experimental test with a car as the target, the system achieved 100% coverage rate and 92.7% accuracy in detecting defects, showing that the proposed method can capture the complete object surface image and detect defects.

關鍵字(中)

★ 無人機
★ 點雲處理
★ 最佳化方法
★ 深度學習

關鍵字(英)

論文目次

摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 viii
第一章緒論 1
1.1 研究背景與動機 1
1.2 文獻回顧 2
1.3 論文目標 3
1.4 論文架構 4
第二章系統架構與軟硬體介紹 5
2.1 系統架構 5
2.2 硬體介紹 6
2.2.1 無人機負載介紹 6
2.2.2 無人機硬體介紹 8
2.2.3 電腦設備介紹 11
2.3 無人機軟韌體介紹 12
2.3.1 韌體 14
2.3.2 rtabmap 14
2.3.3 realsense2_camera 16
2.3.4 mavros 16
2.3.5 SlamToMavros 16
2.3.6 GimbalCamera 17
2.3.7 ManualFlight 17
2.3.8 AutoInspection 17
2.3.9 RClistener 17
2.4 電腦軟體介紹 17
第三章模擬環境建置與修改 19
3.1 模擬環境介紹 19
3.1.1 SITL (Software in the Loop) 19
3.1.2 Unreal Engine 20
3.1.3 AirSim (Aerial Informatics and Robotics Simulation) 21
3.2 模擬環境建置 22
3.3 模擬環境修改 23
3.3.1 修改TF坐標系 24
3.3.2 旋轉雲台取得連續影像 24
第四章路徑規劃與驗證 26
4.1 點雲前處理 26
4.1.1 水平與高度校正 26
4.1.2 切除地平面 28
4.1.3 物件朝向與位置校正 28
4.2 無人機位置與姿態規劃 30
4.2.1 導航點計算 30
4.2.2 偏航與雲台角度計算 33
4.3 路徑規劃 36
4.4 點雲相似度匹配演算法 38
第五章瑕疵偵測網路 41
5.1 資料蒐集 41
5.2 網路架構與訓練 42
第六章實驗結果 45
6.1 路徑規劃 45
6.2 瑕疵偵測網路 46
6.3 戶外測試 48
6.3.1 規劃無人機自動巡檢路徑 49
6.3.2 自動巡檢瑕疵偵測 50
6.3.3 拍攝覆蓋率驗證 53
第七章結論與未來展望 55
7.1 結論 55
7.2 未來展望 55
參考文獻 57
附錄 61

參考文獻

[1] 李固倫, "台灣地區航空維修與檢查人為因素之案例分析," 碩士論文, 國立臺灣科技大學, 2006.
[2] 趙湘龍, "空軍戰術戰鬥機聯隊修護人為誤失及其造成因素之研究," 碩士論文, 國防管理學院, 2005.
[3] D. Waleed, et al., "Drone-based ceramic insulators condition monitoring," IEEE Transactions on Instrumentation and Measurement, vol. 70, 2021.
[4] C. Xu, et al., "Power line-guided automatic electric transmission line inspection system," IEEE Transactions on Instrumentation and Measurement, vol. 71, 2022.
[5] N. M. Shajahan, et al., "Automated inspection of monopole tower using drones and computer vision," International Conference on Intelligent Autonomous Systems (ICoIAS), 2019, pp. 187-192.
[6] E. Salahat, et al., "Waypoint planning for autonomous aerial inspection of large-scale solar farms," The 45th Annual Conference of the IEEE Industrial Electronics Society, 2019, pp. 763-769.
[7] J.-S. Liu and W.-C. Chang, "Vision-based drone navigation for orbital inspection of pole-like objects," IEEE International Conference on Robotic Computing (IRC), 2020, pp. 410-411.
[8] J. Xie, L. R. G. Carrillo, and L. Jin, "Path planning for UAV to cover multiple separated convex polygonal regions," IEEE Access, 2020, vol. 8, pp. 51770-51785.
[9] Z. Shen, et al., "CPPNet: A coverage path planning network," arXiv preprint arXiv:2108.01626, 2021.
[10] A. Sonmez, E. Kocyigit, and E. Kugu, "Optimal path planning for UAVs using genetic algorithm," International Conference on Unmanned Aircraft Systems (ICUAS), 2015, pp. 50-55.
[11] F. Xie, et al., "An adaptive defect detection technology for car-bodies surfaces," Chinese Automation Congress (CAC), 2019, pp. 1023-1028.
[12] J. Guo, et al., "Civil aircraft surface defects detection based on histogram of oriented gradient," IEEE 1st International Conference on Civil Aviation Safety and Information Technology (ICCASIT), 2019, pp. 34-38.
[13] T. Malekzadeh, et al., "Aircraft fuselage defect detection using deep neural networks," arXiv preprint arXiv:1712.09213, 2017.
[14] J. Miranda, et al., "UAV-based inspection of airplane exterior screws with computer vision," The 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2019.
[15] D. Zhang, et al., "Investigation of aircraft surface defects detection based on yolo neural network," The 7th International Conference on Information Science and Control Engineering (ICISCE), 2020, pp. 781-785.
[16] J. Redmon and A. Farhadi, "YOLOv3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[17] S. Ren, et al., "Faster R-CNN: Towards real-time object detection with region proposal networks," arXiv preprint arXiv:1506.01497, 2015.
[18] A. Doğru, et al., "Using convolutional neural networks to automate aircraft maintenance visual inspection," Multidisciplinary Digital Publishing Institute (MDPI) Aerospace, vol. 7, iss. 12, no. 171, 2020.
[19] K. He, et al., "Mask R-CNN," arXiv preprint arXiv:1703.06870, 2017.
[20] S. Pan and Q. Yang, "A survey on transfer learning," IEEE Transactions on Knowledge and Data Engineering, 2010, vol. 22, pp. 1345-1359.
[21] P.-F. Yao, et al., "Research on technology of autonomous inspection system for UAV based on improved YOLOv4," The 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE), 2020, pp. 664-668.
[22] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
[23] "UP Squared迷你電腦," [Online]. Available: https://www.aaeon.com/tw/p/iot-gateway-maker-boards-up-squared. [Accessed: June, 2022].
[24] "Intel® RealSense™ Depth Camera D435i," [Online]. Available: https://www.intelrealsense.com/depth-camera-d435i/. [Accessed: June, 2022].
[25] "配備廣角鏡頭的羅技C930e 1080p 商務網路攝影機," [Online]. Available: https://www.logitech.com/zh-hk/products/webcams/c930e-business-webcam.960-000976.html. [Accessed: June, 2022].
[26] "TAROT T2-2D 2 AXIS GIMBAL," [Online]. Available: http://www.tarotrc.com /Product/Detail.aspx?Lang=en&Id=46034110-6a78-44c9-81f0-c9e899af745d. [Accessed: June, 2022].
[27] "SunnySky X Series V3 X2216 V3 Brushless Motors Short Shaft Version," [Online]. Available: https://sunnyskyusa.com/products/sunnysky-x2216. [Accessed: June, 2022].
[28] "The Cube Orange With ADSB-In Overview," [Online]. Available: https://ardupilot.org /copter/docs/common-thecubeorange-overview.html. [Accessed: June, 2022].
[29] "T10J使用說明書," [Online]. Available: https://www.tw-futaba.com.tw /Portals/0/FTB_Product/RC/Files/TX/T10J%20-%E4%B8%AD%E6%96%87%E8%AA %AA%E6%98%8E%E6%9B%B8(%E7%B9%81%E9%AB%94%E7%89%88).pdf. [Accessed: June, 2022].
[30] "ROG Zephyrus G14 GA401," [Online]. Available: https://rog.asus.com/tw/laptops/rog-zephyrus/rog-zephyrus-g14-series/. [Accessed: June, 2022].
[31] "A2000UA 超世代無線網卡," [Online]. Available: https://totolink.tw /products_spec/A2000UA. [Accessed: June, 2022].
[32] "Open Source Autopilot for Drones - PX4 Autopilot," [Online]. Available: https://px4.io/. [Accessed: June, 2022].
[33] "ArduPilot - Versatile, Trusted, Open," [Online]. Available: https://ardupilot.org/. [Accessed: June, 2022].
[34] "ArduCopter," [Online]. Available: https://ardupilot.org/copter/. [Accessed: June, 2022].
[35] "ArduPlane," [Online]. Available: https://ardupilot.org/plane/. [Accessed: June, 2022].
[36] "ArduRover," [Online]. Available: https://ardupilot.org/rover/. [Accessed: June, 2022].
[37] "ArduSub," [Online]. Available: https://www.ardusub.com/. [Accessed: June, 2022].
[38] N. Altuntaş, et al., "Comparison of 3-dimensional SLAM systems: RTAB-Map vs. Kintinuous," International Conference on Computer Science and Engineering (UBMK), 2017, pp. 99-103.
[39] A. Merzlyakov and S. Macenski, "A comparison of modern general-purpose visual SLAM approaches," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021.
[40] M. Adorjan, "OpenSfM : A collaborative structure-from-motion system," Diploma Thesis, Vienna University of Technology, 2016.
[41] S. Shah, et al., "AirSim: High-fidelity visual and physical simulation for autonomous vehicles," arXiv preprint arXiv:1705.05065, 2017.
[42] "Voxel Downsampling - Open3D," [Online]. Available: http://www.open3d.org/docs /0.8.0/tutorial/Basic/pointcloud.html. [Accessed: June, 2022].
[43] 方紹宇, "取締違規停車之自主巡邏無人機," 碩士論文, 國立中央大學, 2020.
[44] M. Tappe, et al., "Autonomous surface inspection of airplanes with unmanned aerial systems," The 7th International Conference on Automation, Robotics and Applications (ICARA), 2021, pp. 135-139.
[45] "Travelling Salesman Problem – Wikipedia," [Online]. Available: https://en.wikipedia.org /wiki/Travelling_salesman_problem. [Accessed: June, 2022].
[46] David L., "Applying adaptive algorithms to epistatic domains," The 9th International Joint Conference on Artificial Intelligence, 1985, vol. 1, pp. 162-164.
[47] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, "Scaled-YOLOv4: Scaling cross stage partial network," IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 13024-13033.
[48] C.-Y. Wang, et al., "CSPNet: A new backbone that can enhance learning capability of CNN," IEEE Conference on Computer Vision and Pattern Recognition Workshop (CVPR Workshop), 2020.
[49] "Car Damage Assessment - Kaggle," [Online]. Available: https://www.kaggle.com /datasets/hamzamanssor/car-damage-assessment. [Accessed: June, 2022].

指導教授

王文俊

審核日期

2022-9-1

推文