基於AI技術之視障人士的行進避障及超商辨識與引導

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：85

、訪客IP：3.145.98.11

姓名

謝易軒(I-Hsuan Hsieh) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

基於AI技術之視障人士的行進避障及超商辨識與引導

相關論文

★ 直接甲醇燃料電池混合供電系統之控制研究	★ 利用折射率檢測法在水耕植物之水質檢測研究
★ DSP主控之模型車自動導控系統	★ 旋轉式倒單擺動作控制之再設計
★ 高速公路上下匝道燈號之模糊控制決策	★ 模糊集合之模糊度探討
★ 雙質量彈簧連結系統運動控制性能之再改良	★ 桌上曲棍球之影像視覺系統
★ 桌上曲棍球之機器人攻防控制	★ 模型直昇機姿態控制
★ 模糊控制系統的穩定性分析及設計	★ 門禁監控即時辨識系統
★ 桌上曲棍球：人與機械手對打	★ 麻將牌辨識系統
★ 相關誤差神經網路之應用於輻射量測植被和土壤含水量	★ 三節式機器人之站立控制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文旨在設計一穿戴式導盲裝置，建立一系列的演算法，引導視障人士於戶外行動，以確保其安全的行走在人行道與斑馬線上，並且確保視障者能夠安全的通過馬路，除此之外，當抵達之目的地設定為四大超商時，能夠辨識超商招牌並引導視障者走近超商的門口前。硬體的部分使用Nvidia推出的Jetson AGX Xavier嵌入式系統板，搭配Stereolabs ZED 2雙目攝影機組合而成，將ZED 2安裝在使用者配戴的鴨舌帽沿上，並將嵌入式系統板與其它硬體配件固定於木板上，讓使用者背在後面，採用一顆5200mAh的鋰電池為電力供給，能夠提供約連續2.5小時的使用。軟體的部分使用經過預訓練的Fast-SCNN語義分割模型，結合視障者正前方影像的語義分割結果，以及ZED2輸出的深度圖，設計一可行走區域檢測演算法，將畫面切分成七個方向，計算不同方向的可行走程度，選擇出一個可行走方向，並輸出語音提示，引導使用者行走較為空曠安全的方向的人行道或斑馬線上，同時實現避障的功能。在使用者通過馬路前，使用Invignal平台提供之API來讀取紅綠燈狀態，以語音提示的方式讓使用者可以知道紅綠燈的燈號狀態以及剩餘秒數，引導使用者安全行走斑馬線通過馬路。當目的地設定為四大超商時，利用經過預訓練之YOLOv5模型，結合超商招牌的辨識結果與ZED2的地磁資訊，將招牌從影像畫面中的位置轉換到實體空間中的地磁方位，計算使用者與目標的朝向角與距離，以不同頻率的音效提示使用者靠近超商。研究結果顯示，根據本論文所提出的避障引導機制，能夠正確且安全的在人行道與斑馬線上行走，且確保在排除語音提示的延遲下，本演算法每秒可以處理至少6幀以上的畫面。在實際與視障者實驗後，證實本研究所提出的方法能夠正確的指引視障者行走在人行道及斑馬線上，且正確且成功地避開障礙物，紅綠燈號狀態提示有效增加視障者通過路口時的安全性，最後能安全地抵達所設定之目的地。

摘要(英)

This study aims to design a wearable device for visually impaired and establish a series of algorithms for outdoor walking. The main purpose is to guide the visually impaired to walk on sidewalks or crosswalks correctly, and ensure that visually impaired can pass through intersections safely. The hardware of this study uses the Jetson AGX Xavier embedded system board combined with the Stereolabs ZED 2 RGB-D camera. The camera ZED 2 is mounted on the hat brim and the embedded system board and other hardware accessories are carried on the back of the user. Furthermore, a 5200mAh lithium battery is used as the power supply of the device and ensure 2.5 hours running time. The software is a pre-trained Fast-SCNN semantic segmentation model. By combining with the segmentation result and the depth map obtained by ZED 2, a walkable area detection algorithm is proposed to calculate the walkability confidence degree for the image in front of the visually impaired person. Then, according to the result of the walkability confidence degree, the guiding voice is given to guide the suitable forward direction to the user. In addition, when the user is passing a crosswalk, the API provided by the Invignal platform is used to learn of the traffic light status and provide the seconds of red light such that the user can pass the crosswalk safely. Finally, a pre-trained YOLOv5 model is applied to recognize the signboards of convenience stores. Based on the magnetometer measurement by ZED 2 and the image in the screen so that we can calculate the user position relative to the convenience store and guide the visually impaired person approach the convenience store gate. Finally there was an experiment with a visually impaired person, the experiment showed that the proposed algorithm and the wearable device are effective to guide the visually impaired people to walk on sidewalks correctly and passed through the crosswalk safely.

關鍵字(中)

★ 穿戴式裝置
★ 導盲輔具
★ 深度學習
★ 電腦視覺
★ 避障引導

關鍵字(英)

★ wearable device
★ visually impaired
★ deep learning
★ computer vision
★ obstacle avoidance

論文目次

目錄
摘要 i
Abstract ii
目錄 iv
圖目錄 vi
表目錄 viii
第一章、緒論 1
1.1 研究背景與動機 1
1.2 文獻探討與回顧 2
1.3 論文目標 3
1.4 論文架構 4
第二章、軟硬體系統架構介紹 5
2.1 硬體介紹 5
2.1.1 嵌入式系統開發板 5
2.1.2 深度攝影機鏡頭 6
2.1.3 電池 6
2.1.4 音效卡 7
2.1.5 4G（LTE）模組 8
2.1.6 實體配置 9
2.2 系統架構 9
2.2.1 各子計畫功能與整合架構 10
2.2.2 室外功能整合架構 11
2.3 ZED SDK串流 12
2.4 室外功能切換流程 12
第三章、避障引導機制 14
3.1 避障引導機制架構 14
3.2 七個方向的可行走程度計算 15
3.2.1 輸入資料 15
3.2.2 以語義分割為基礎的可行走程度計算 17
3.2.3 以深度圖為基礎的可行走程度計算 19
3.2.4 合併考慮可行走程度計算 22
3.3 方向選擇與語音提示 23
3.4 未直視前方提示 24
3.5 路面情況判斷機制 26
3.6 語音提示管理器 27
第四章、紅綠燈號狀態 28
4.1 Invignal 28
4.2 紅綠燈號狀態 30
第五章、超商招牌靠近機制 32
5.1 招牌靠近機制架構 32
5.2 招牌定位 33
5.3 招牌選擇與狀態決策 34
第六章、實驗結果 38
6.1 避障引導機制 38
6.1.1 人行道情況 38
6.1.2 斑馬線情況 45
6.2 超商招牌靠近機制 52
6.2.1 招牌定位 52
6.2.2 招牌選擇機制 53
第七章、結論與未來展望 54
7.1 結論 54
7.2 未來展望 55
參考文獻 57
研究投稿 61

參考文獻

參考文獻
[1] R. Bourne, et al., "Trends in prevalence of blindness and distance and near vision impairment over 30 years: an analysis for the global burden of disease study," Lancet Global Health, vol. 9, no. 2, pp. e130-e143, Feb. 2021.
[2] "衛生福利部統計處," [Online]. Available: https://dep.mohw.gov.tw/DOS/cp-2976-13815-113.html. [Accessed: June, 2020].
[3] A. Riazi, et al., "Outdoor difficulties experienced by a group of visually impaired Iranian people," Journal of current ophthalmology, vol. 28, pp. 85-90, June. 2016.
[4] G. P. Soong, J. E. Lovie-Kitchin, and B. Brown, "Does mobility performance of visually impaired adults improve immediately after orientation and mobility training?," Optometry and Vision Science, vol. 78, no. 9, pp. 657-666, Sept. 2001.
[5] R. Pyun, Y. Kim, P. Wespe, R. Gassert, and S. Schneller, "Advanced augmented white cane with obstacle height and distance feedback, " IEEE International Conference on Rehabilitation Robotics, (ICORR), pp. 1–6, Jun. 2013.
[6] L. M. Tomkins et al., "Behavioral and physiological predictors of guide dog success, " Journal of Veterinary Behavior: Clinical Applications and Research, 2011, vol. 6, pp. 178-187.
[7] J. Bai, D. Liu, G. Su, and Z. Fu, "A cloud and vision-based navigation system used for blind people," International Conference on Artificial Intelligence Automation Control Technologies (AIACT), Wuhan, China, 2017, pp. 127–162.
[8] L. Whitmarsh, "The benefits of guide dog ownership," Visual Impairment Research, vol. 7, no. 1, pp. 27–42, 2005.
[9] 邱文欣, "基於深度學習之單眼距離估測與機器人戶外行走控制," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2019.
[10] 汪孟璇, "基於深度學習之道路資訊辨識導盲系統," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2020.
[11] 沈鴻儒, "基於深度學習之道路障礙物偵測與盲人行走輔助技術," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2020.
[12] 城筱筑, "基於AI技術之視障人士的路況分析及障礙物辨識與測距," 碩士, 電機工程學系, 國立中央大學, 桃園市, 2021.
[13] J. M. Sáez, F. Escolano, and M. A. Lozano, "Aerial obstacle detection with 3-d mobile devices, " IEEE Journal of Biomedical and Health Informatics, vol. 19, no. 1, pp. 74-80, Jan. 2015.
[14] W.-J. Chang, L.-B. Chen, M.-C. Chen, J.-P. Su, C.-Y. Sie, and C.-H. Yang, "Design and implementation of an intelligent assistive system for visually impaired people to aerial obstacles avoidance and fall detection," IEEE Sensors Journal, vol. 20, no. 17, pp. 10199–10210, Sep. 2020.
[15] W. M. Elmannai and K. M. Elleithy, "A highly accurate and reliable data fusion framework for guiding the visually impaired," IEEE Access, vol. 6, pp. 33029-33054, 2018.
[16] D. Croce, L. Giarre, F. Pascucci, I. Tinnirello, G. E. Galioto, D. Garlisi, and A. Lo Valvo, "An indoor and outdoor navigation system for visually impaired people," IEEE Access, vol. 7, pp. 170406-170418, 2019.
[17] A. Aladrén, G. López-Nicolás, L. Puig, and J. J. Guerrero, "Navigation assistance for the visually impaired using RGB-D sensor with range expansion," IEEE Systems Journal, vol. 10, no. 3, pp. 922-932, Sept. 2016.
[18] A. Paszke, A. Chaurasia, S. Kim, and E. Culurciello, "Enet: A deep neural network architecture for real-time semantic segmentation," arXiv preprint arXiv:1606.02147, 2016.
[19] R. P. Poudel, S. Liwicki, and R. Cipolla, "Fast-Scnn: Fast semantic segmentation network," arXiv preprint arXiv:1902.04502, 2019.
[20] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
[21] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[22] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," IEEE Conference on Computer Vision Pattern Recognition (CVPR), Jun. 2016, pp. 779–788.
[23] J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," arXiv preprint arXiv:1612.08242, 2016.
[24] S. Ren, K. He, R. Girshick and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, June 2017.
[25] R. Girshick, J. Donahue, T. Darrell and J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 580-587.
[26] R. Girshick, "Fast R-CNN," in IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1440-1448.
[27] J. R. Uijlings, K. E. Van De Sande, T. Gevers and A. W. Smeulders, "Selective Search for Object Recognition," International Journal of Computer Vision, 2013, pp. 154-171.
[28] Bochkovskiy, Alexey, Chien-Yao Wang, and Hong-Yuan Mark Liao, "Yolov4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
[29] R. Jocher et al., "yolov5: v5.0," 2020. [Online]. Available: https://github.com/ultralytics/yolov5
[30] "Nvidia Jetson AGX Xavier", [Online]. Available: https://www.nvidia.com/zh-tw/autonomous-machines/embedded-systems/jetson-agx-xavier/. [Accessed: June, 2021].
[31] "Stereolabs ZED", [Online]. Available: https://www.stereolabs.com/zed/ [Accessed: June, 2021].
[32] "Stereolabs ZED 2, " [Online]. Available: https://www.stereolabs.com/zed-2/. [Accessed: June, 2021].
[33] “萬用 AC 行動電源 enerpad AC42K," [Online]. Available: https://www.enerpad.com.tw/product/10715e6a-83ad-4195-a586-fe2aba024e42. [Accessed: June, 2021].
[34] “Desire Power V8 14.8V 5200mAh 35C-70C 4S鋰電池," [Online]. Available: https://shopee.tw/-%E6%90%9E%E5%95%A5%E9%A3%9B%E6%A9%9F-Desire-Power-V8-14.8V-5200mAh-35C-70C-4S%E9%8B%B0%E9%9B%BB%E6%B1%A0XT60-BSMI%E8%AA%8D%E8%AD%89-i.17393576.2037112384. [Accessed: June, 2021].
[35] “HAGiBiS海備思USB鋁合金外接音效卡," [Online]. Available: https://24h.pchome.com.tw/prod/DCAC1S-A900AOOTG. [Accessed: June, 2021].
[36] “SIM7600CE 4G HAT用戶手冊," [Online]. Available: https://www.waveshare.net/w/upload/7/73/SIM7600CE-4G-HAT-Manual-CN_.pdf. [Accessed: June, 2021].
[37] "Invignal 號誌領航先導技術," [Online]. Available: http://www.invignal.com/. [Accessed: June, 2021].
[38] 王文俊, "認識Fuzzy理論與應用第四版," 全華科技圖書公司, 2017.
[39] "Position Tracking Overview," [Online]. Available: https://www.stereolabs.com/docs/positional-tracking/. [Accessed: June, 2021].
[40] "Coordinate Frames," [Online]. Available: https://www.stereolabs.com/docs/positional-tracking/coordinate-frames/. [Accessed: June, 2021].

指導教授

王文俊(Wen-June Wang)

審核日期

2021-7-6

推文