基於深度學習之道路資訊辨識導盲系統

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：83

、訪客IP：18.117.70.24

姓名

汪孟璇(Meng-Hsuan Wang) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

基於深度學習之道路資訊辨識導盲系統

相關論文

★ 直接甲醇燃料電池混合供電系統之控制研究	★ 利用折射率檢測法在水耕植物之水質檢測研究
★ DSP主控之模型車自動導控系統	★ 旋轉式倒單擺動作控制之再設計
★ 高速公路上下匝道燈號之模糊控制決策	★ 模糊集合之模糊度探討
★ 雙質量彈簧連結系統運動控制性能之再改良	★ 桌上曲棍球之影像視覺系統
★ 桌上曲棍球之機器人攻防控制	★ 模型直昇機姿態控制
★ 模糊控制系統的穩定性分析及設計	★ 門禁監控即時辨識系統
★ 桌上曲棍球：人與機械手對打	★ 麻將牌辨識系統
★ 相關誤差神經網路之應用於輻射量測植被和土壤含水量	★ 三節式機器人之站立控制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文旨在對現有之導盲機器人進行改良，並提出一穿戴式裝置，以協助視障人士安全地於戶外行走。第一部分為對導盲機器人進行以下三點改進：第一點是更換用於辨識機器人前方影像中，道路與障礙物區域範圍的深度學習網路，同時蒐集更多訓練資料並對其進行資料擴增，藉此提升道路、人與車的辨識準確率。第二點為新增當前方可行走空間不足時，機器人將原地旋轉以尋找其他路線之功能。第三點則是當視障者欲前往之目的地為店家時，機器人將利用其他子團隊之招牌辨識系統於目的地附近尋找該招牌、計算其與機器人間之距離與夾角，本研究藉此操控機器人以帶領視障者更靠近店家。
　　第二部分為把導盲機器人簡化成穿戴式裝置，此裝置同樣具有以深度學習網路辨識環境資訊之功能，並新增辨識班馬線、帶領視障者走在斑馬線上以通過路口之演算法。該演算法結合深度學習之辨識結果與影像處理之方法偵測前方是否有斑馬線，並於找到斑馬線的位置後，帶領視障者靠近並對齊之，以達到協助視障者沿斑馬線通過馬路之目標。
　　引導視障者的控制指令將經由手機之語音播報系統通知視障者，此播報系統亦會提醒視障者正確行進方向與前方障礙物之距離與種類等內容。而為避免播放之訊息互相干擾，本論文將定義各內容之優先權，並讓優先權較高之資訊具有打斷正在播報內容之權利，以確保重要資訊能優先被播出。
　　簡而言之，本論文改良後之導盲機器人的環境辨識準確率提升、於無路可走時將自動找路且可帶領視障者靠近店家；而於穿戴式裝置上開發的過馬路輔助系統則具有辨識斑馬線、引導視障者正確地走在斑馬線上以安全通過路口之功能。

摘要(英)

This thesis aims to improve the guiding robot and develop a wearable device for helping visually impaired people to walk outdoor safely. The guiding robot has three improvements. First, this research uses much more training data and another semantic segmentation neural network to raise the accuracy of recognizing road, people, and cars. Second, the robot is added one more function that it will rotate to find another way if its walkable area is not enough to go through. Third, while the destination is a store, the robot will lead the blind closer to the store due to the direction information between the robot and the store signboard provided from other research team.
　　Next, the guiding robot will be changed to a wearable device. The wearable device has not only the same functions as the robot, but also the new algorithm which can detect the position of the crosswalks so the blind can cross the road by following the crosswalk.
　　The information including the guiding commands which used to guide the blind and the distances and types of obstacles in front of him/her will be broadcasted by a smartphone. To avoid interference between broadcasted messages, this research defines the priority of each message. The messages with higher priority can interrupt the broadcasting one to ensure the important information will not be missed.
　　In conclusion, the thesis improves the accuracy of environment recognition, has the abilities of route finding and can take the blind to the store. The wearable device has the same functions as the robot and can detect crosswalks in order to lead the visually impaired people pass the intersection safely.

關鍵字(中)

★ 穿戴式裝置
★ 導盲輔具
★ 深度學習
★ 斑馬線偵測

關鍵字(英)

★ wearable device
★ blind aids
★ deep learning
★ crosswalk detection

論文目次

摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 ix
第一章　緒論 1
　1.1　研究動機與背景 1
　1.2　文獻回顧 1
　1.3　論文目標 4
　1.4　論文架構 4
第二章　系統架構與硬體介紹 5
　2.1　系統架構與通訊格式 5
　　2.1.1　導盲機器人 6
　　2.1.2　穿戴式裝置 7
　2.2　硬體介紹 8
第三章　語義分割網路 14
　3.1　網路架構 14
　3.2　訓練資料 17
第四章　斑馬線偵測演算法 22
　4.1　斑馬線偵測 22
　4.2　追線機制 30
第五章　手機導航系統 32
　5.1　導航系統 32
　　5.1.1　導盲機器人 34
　　5.1.2　穿戴式裝置 36
　5.2　語音播報 38
第六章　戶外控制系統 41
　6.1　導盲機器人 41
　　6.1.1　目的地模式 42
　　6.1.2　商店模式 43
　6.2　穿戴式裝置 44
　　6.2.1　參數介紹 45
　　6.2.2　路口判斷 46
　　6.2.3　右轉 48
　　6.2.4　直走 49
　　6.2.5　左轉 51
第七章　實驗結果 52
　7.1　語義分割 52
　7.2　導盲機器人 53
　　7.2.1　停止/找路機制 54
　　7.2.2　商店模式 54
　7.3　斑馬線偵測 56
　7.4　穿戴式裝置之過馬路輔助系統 57
　　7.4.1　右轉 57
　　7.4.2　直走 59
　　7.4.3　左轉 61
第八章　結論與未來展望 68
　8.1　結論 68
　8.2　未來展望 69
參考文獻 71

參考文獻

[1] "衛生福利部統計處," [Online]. Available: https://dep.mohw.gov.tw/DOS/cp-2976-13815-113.html. [Accessed: June, 2020].
[2] 邱文欣, "基於深度學習之單眼距離估測與機器人戶外行走控制," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2019.
[3] 廖浤鈞, "基於深度學習之關聯式追蹤網路," 碩士, 資訊工程學系, 國立中央大學, 桃園市, 2020.
[4] 沈鴻儒, "基於深度學習之道路障礙物偵測與盲人行走輔助技術," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2020.
[5] A. Paszke, A. Chaurasia, S. Kim, and E. Culurciello, "Enet: A deep neural network architecture for real-time semantic segmentation," arXiv preprint arXiv:1606.02147, 2016.
[6] C. Godard, O. Mac Aodha, and G. J. Brostow, "Unsupervised monocular depth estimation with left-right consistency," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, 2017, pp. 270-279.
[7] H. C. Wang, R. K. Katzschmann, S. Teng, B. Araki, L. Giarré, and D. Rus, "Enabling independent navigation for visually impaired people through a wearable vision-based feedback system," IEEE International Conference on Robotics and Automation(ICRA), Singapore, 2017, pp. 6533-6540.
[8] H. Badino, U. Franke, and D. Pfeiffer, "The stixel world-a compact medium level representation of the 3d-world," 31st DAGM Symposium on Pattern Recognition, Jena, Germany, 2009, pp. 51-60.
[9] Y. Zhang, Y. Zhao, T. Wei, and J. Chen, "Dynamic path planning algorithm for wearable visual navigation system based on the improved A*," IEEE International Conference on Imaging Systems and Techniques(IST), Beijing, China, 2017, pp. 1-6.
[10] K. Yang, L. M. Bergasa, E. Romera, R. Cheng, T. Chen, and K. Wang, "Unifying terrain awareness through real-time semantic segmentation," IEEE Intelligent Vehicles Symposium(IV), Suzhou, China, 2018, pp. 1033-1038.
[11] J. Bai, Z. Liu, Y. Lin; Y. Li; S. Lian, and D. Liu, "Wearable travel aid for environment perception and navigation of visually impaired people," Electronics, vol. 8, no. 6, pp. 697, 2019.
[12] J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, "Smart guiding glasses for visually impaired people in indoor environment," IEEE Transactions on Consumer Electronics, vol. 63, no. 3, pp. 258-266, 2017.
[13] J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, "Virtual-blind-road following-based wearable navigation device for blind people," IEEE Transactions on Consumer Electronics, vol. 64, no. 1, pp. 136-143, 2018.
[14] C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang, "Bisenet: Bilateral segmentation network for real-time semantic segmentation," European Conference on Computer Vision(ECCV), Munich, Germany, 2018, pp. 325-341.
[15] E. Romera, J. M. Álvarez, L. M. Bergasa, and R. Arroyo, "ERFNet: Efficient residual factorized convnet for real-time semantic segmentation," IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 1, pp. 263-272, 2018.
[16] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Las Vegas, Nevada, 2016, pp. 770-778.
[17] J. M. Álvarez and L. Petersson, "DecomposeMe: Simplifying convnets for end-to-end learning," arXiv preprint arXiv:1606.05426, 2016.
[18] H. Zhao, X. Qi, X. Shen, J. Shi, and J. Jia, "ICNet for real-time semantic segmentation on high-resolution images," European Conference on Computer Vision(ECCV), Munich, Germany, 2018, pp. 405-420.
[19] R. P. Poudel, S. Liwicki, and R. Cipolla, "Fast-Scnn: Fast semantic segmentation network," arXiv preprint arXiv:1902.04502, 2019.
[20] H. Li, P. Xiong, H. Fan, and J. Sun, "DFANet: Deep feature aggregation for real-time semantic segmentation," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, California, 2019, pp. 9522-9531.
[21] J. Coughlan and H. Shen, "A fast algorithm for finding crosswalks using figure-ground segmentation," 2nd Workshop on Applications of Computer Vision in conjunction with ECCV, Graz, Austria, 2006.
[22] J. Choi, B. T. Ahn, and I. S. Kweon, "Crosswalk and traffic light detection via integral framework," The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, Incheon, Korea, 2013, pp. 309-312.
[23] S. Mascetti, L. Picinali, A. Gerino, D. Ahmetovic, and C. Bernareggi, "Sonification of guidance data during road crossing for people with visual impairments or blindness," International Journal of Human-Computer Studies, vol. 85, pp. 16-26, 2016.
[24] D. Ahmetovic, C. Bernareggi, A. Gerino, and S. Mascetti, "ZebraRecognizer: Efficient and precise localization of pedestrian crossings," 22nd International Conference on Pattern Recognition(ICPR), Stockholm, Swedish, 2014, pp. 2566-2571.
[25] Y. Zhai, G. Cui, Q. Gu, and L. Kong, "Crosswalk detection based on MSER and ERANSAC," IEEE International Conference on Intelligent Transportation Systems, Las Palmas, Spain, 2015, pp. 2770-2775.
[26] J. Matas, O. Chum, M. Urban, and T. Pajdla, "Robust wide baseline stereo from maximally stable extremal regions," 13th British Machine Vision Conference, Cardiff, UK, pp. 384-393, 2002.
[27] M. Fischler and R. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, vol. 24, no. 6, pp. 381-395, 1981.
[28] R. Cheng, K. Wang, K. Yang, N. Long, W. Hu, H. Chen, J. Bai, and D. Liu, "Crosswalk navigation for people with visual impairments on a wearable device," Journal of Electronic Imaging, vol. 26, no. 5, pp. 053025, 2017.
[29] V. Tümen and B. Ergen, "Intersections and crosswalk detection using deep learning and image processing techniques," Physica A: Statistical Mechanics and its Applications, vol. 543, 2019.
[30] "Jetson AGX Xavier," [Online]. Available: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/. [Accessed: June, 2020].
[31] "Intel Dual Band Wireless-AC 8265 NGW," [Online]. Available: https://ark.intel.com/content/www/tw/zh/ark/products/94150/intel-dual-band-wireless-ac-8265.html. [Accessed: June, 2020].
[32] "Samsung Galaxy S9 智慧型手機," [Online]. Available: https://www.samsung.com/tw/support/mobile-devices/what-are-the-new-samsung-galaxy-s9-and-s9-plus-specs/. [Accessed: June, 2020].
[33] "Logitech C930e 網路攝影機," [Online]. Available: https://www.logitech.com/zh-tw/product/c930e-webcam. [Accessed: June, 2020].
[34] "ZED," [Online]. Available: https://www.stereolabs.com/zed/. [Accessed: June, 2020].
[35] "Ublox-NEO-M8N," [Online]. Available: https://www.u-blox.com/en/product/neo-m8-series#tab-documentation-resources. [Accessed: June, 2020].
[36] "AQMD6010BLS 直流無刷馬達驅動器," [Online]. Available: http://www.akelc.com/BLDCMotor/show_78.html. [Accessed: June, 2020].
[37] "80BL110S50 直流無刷馬達," [Online]. Available: https://detail.1688.com/offer/1272236959.html. [Accessed: June, 2020].
[38] "萬用AC行動電源 enerpad AC42K," [Online]. Available: https://www.enerpad.com.tw/product/10715e6a-83ad-4195-a586-fe2aba024e42. [Accessed: June, 2020].
[39] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, "MobileNets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv: 1704.04861, 2017.
[40] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, "MobileNetV2: Inverted residuals and linear bottlenecks," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Salt Lake City, Utah, 2018, pp. 4510-4520.
[41] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, "Pyramid scene parsing network," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, 2017, pp. 2881-2890.
[42] 賴怡靜, "基於深度學習之距離估測與自動避障的戶外導航機器人," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2018.
[43] N. Otsu, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, vol. smc-9, no. 1, pp. 62-66, 1979.
[44] P. V. C. Hough, "Method and means for recognizing complex patterns," 1962.
[45] E. W. Weisstein, "Least squares fitting-Perpendicular offsets," [Online]. Available: https://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html. [Accessed: June, 2020].
[46] "Haversine formula," [Online]. Available: https://en.wikipedia.org/wiki/Haversine_formula. [Accessed: June, 2020].
[47] "Spherical trigonometry," [Online]. Available: https://en.wikipedia.org/wiki/Spherical_trigonometry. [Accessed: June, 2020].

指導教授

王文俊(Wen-June Wang)

審核日期

2020-7-21

推文