基於深度學習之室內盲人引導輔助系統

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：20

、訪客IP：3.22.241.171

姓名

王婷(Ting Wang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於深度學習之室內盲人引導輔助系統
(A Deep-learning-based Indoor Navigation Assistance System for Visually Impaired Persons)

相關論文

★ 以Q-學習法為基礎之群體智慧演算法及其應用	★ 發展遲緩兒童之復健系統研製
★ 從認知風格角度比較教師評量與同儕互評之差異：從英語寫作到遊戲製作	★ 基於檢驗數值的糖尿病腎病變預測模型
★ 模糊類神經網路為架構之遙測影像分類器設計	★ 複合式群聚演算法
★ 身心障礙者輔具之研製	★ 指紋分類器之研究
★ 背光影像補償及色彩減量之研究	★ 類神經網路於營利事業所得稅選案之應用
★ 一個新的線上學習系統及其於稅務選案上之應用	★ 人眼追蹤系統及其於人機介面之應用
★ 結合群體智慧與自我組織映射圖的資料視覺化研究	★ 追瞳系統之研發於身障者之人機介面應用
★ 以類免疫系統為基礎之線上學習類神經模糊系統及其應用	★ 基因演算法於語音聲紋解攪拌之應用

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

視障者要獨立行走在一個陌生且複雜的室內公共空間中，是一件很困難的事情，因此，要如何獲取環境資訊以讓視障者可以在無需旁人輔助的情況下到達目的地，是個重要的研究課題。因此，本論文使用影像處理技術與深度學習做結合，開發出一套室內盲人引導輔助系統，讓視障者能在陌生的環境中獨立行走。
本篇論文所開發出的室內引導盲人輔助系統包含三部分:(1)指標偵測；使用YOLOv3模型搭配深度影像，偵測出室內常見的指標以及地上危險指標，並且計算其距離位置、(2)文字偵測與辨識:使用PSENET模型搭配OCR文字辨識API，偵測與識別指示標示牌中的文字、和(3)方向與指示標示牌資訊配對:使用YOLOv3模型偵測出區域，以配對指示標示牌中的箭頭與資訊。綜合上述三部分之功能，在取得指示標示牌影像後，可以識別此標示牌之資訊，可以透過語音提示的方式來輔助視障者掌握前方陌生的環境資訊。
本系統研發的目的在於輔助視障者在陌生環境公共空間中，提供眼前的資訊，以幫助視障者抵達目的地。實驗結果顯示，平均偵測到的指標準確率達到93%，方向與指示標誌牌資訊配對的精確度為86%，有此可以證明本系統具備一定程度之可用性。

關鍵字：深度學習、導盲系統、室內引導、偵測系統、視障者

摘要(英)

It is difficult for the visually impaired to walk independently in a strange and complicated indoor public space. Therefore, how to obtain environmental information so that the visually impaired can reach the destination without the assistance of others is an important research topic. Therefore, this paper uses image processing technology combined with deep learning to develop an indoor blind navigation assistance system that allow the visually impaired to walk independently in unfamiliar environments.
The indoor blind navigation assistance system developed in this paper consists of three parts: (1) Sign detection: using the YOLOv3 model with depth images to detect common indoor signs and ground hazard indicators, and calculate their distance and location, (2) Word detection and recognition: using PSENET model with the OCR text recognition API to detect and recognize the text in the indicator plate. (3) direction and indicator information pairing: Using YOLOv3 model to detect the area and match the direction indicating arrows and information. Combining the functions of the above three parts, after obtaining the image of the front signboards, the information of the environment can be identified, and the system can assist the visually impaired to grasp the unfamiliar environment information in front via voiced signals.
The purpose of the development of this system is to assist visually impaired people to provide information in front of them in a public space in an unfamiliar environment to help the visually impaired reach their destination. The experimental results show that the average accuracy of the detected indicators reaches 93%, and the accuracy of direction and sign information matching is 86%. From the experimental results, it proves that the system has a certain degree of usability.

Keywords: deep learning, navigation systems for the blind persons, indoor navigation, detection system, visually impaired

關鍵字(中)

★ 深度學習
★ 導盲系統
★ 室內引導
★ 偵測系統
★ 視障者

關鍵字(英)

★ deep learning
★ navigation systems for the blind persons
★ indoor navigation
★ detection system
★ visually impaired

論文目次

摘要 i
ABSTRACT ii
致謝 iv
目錄 v
圖目錄 vii
表目錄 ix
第一章、緒論 1
1-1 研究動機 1
1-2 研究目的 2
1-3 論文架構 3
第二章、相關研究 4
2-1 盲人引導輔具 4
2-1-1 傳統輔具 4
2-1-2 科技輔具產品 6
2-2 室內定位與導航技術 9
2-3 物件偵測模型 13
2-3-1 YOLO 13
2-3-2 YOLOv2 16
2-3-3 YOLOv3 18
2-4 PSENET文字偵測模型 20
第三章、研究方法 22
3-1 硬體介紹 22
3-1-1 NAVIDIA Jetson AGX Xavier嵌入式系統 22
3-1-2 Zed Camera 23
3-2 系統介紹 26
3-2-1 系統架構 26
3-2-2 系統設計 27
3-2-3 系統流程 28
3-3 識別指標模組 30
3-3-1 指標偵測 30
3-3-2 文字偵測與辨識 33
3-3-3 方向與指示標示資訊配對 35
第四章、實驗設計與結果 40
4-1 標誌偵測實驗 40
4-1-1 評估方式 40
4-1-2 IOU閥值實驗結果與分析 43
4-2 標誌偵測資料集解析度影響之實驗 46
4-2-1 實驗設計 46
4-2-2 實驗結果與分析 47
4-3 方向標誌配對實驗 49
4-3-1 評估方式 49
4-3-2 PSENET與YOLOv3比較 51
4-3-3 影像閥值化實驗結果與分析 54
4-4 識別指標模組測試 55
第五章、結論與未來展望 57
5-1 結論 57
5-2 未來展望 58
參考文獻 59

參考文獻

[1] "伊甸福利基金會," [Online]. Available: http://light.eden.org.tw/service_assure_dissemination.htm. [Accessed 22 6 2020].
[2] 余. 唐. 曾亮, "關懷無障礙設施系列報導(一)," [Online]. Available: http://www.arch.net.tw/modern/month/338/338-2.htm. [Accessed 22 6 2020].
[3] "大專校院資源教室輔導人員特教知能(初階)研習講義26," [Online]. Available: http://sencir.spc.ntnu.edu.tw/site/c_file/a_download/t_key/1178. [Accessed 22 6 2020].
[4] "清大盲友會," [Online]. Available: https://blind.cs.nthu.edu.tw/guide/guide-1.html. [Accessed 9 6 2020].
[5] "財團法人愛犬基金會," [Online]. Available: https://www.tfb.org.tw/web/service/service.jsp?no=CP1497412035614. [Accessed 9 6 2020].
[6] L. Chang, "這個新的AR技術能透過讓物品說話幫助盲人辨識方位," [Online]. Available: https://chinese.engadget.com/2018/11/29/ar-headset-tech-helps-blind-navigate/. [Accessed 9 6 2020].
[7] "HoloLens (第一代)硬體," 2019. [Online]. Available: Available: https://docs.microsoft.com/zh-tw/hololens/hololens1-hardware. [Accessed 22 6 2020].
[8] 蔡騰輝, "改善傳統視障輔具南台科大推出物聯網手杖," 2018. [Online]. Available: https://www.digitimes.com.tw/iot/article.asp?cat=158&id=0000543796_PVJ8XV6N8X4UGB7O7LM49. [Accessed 9 6 2020].
[9] 楊晨欣, "Google 新APP要幫視覺障礙人士看世界," [Online]. Available: https://www.bnext.com.tw/article/49067/google-lookout-app-for-visual-impaired-people. [Accessed 9 6 2020].
[10] C. Yang and H. SHao, "Wifi-based indoor positioning," IEEE Communications Magazine, pp. 150-157, 2016.
[11] S. Willis and A. (Sumi) Helal., "RFID information grid for blind navigation and wayfinding," ISWC, vol. 5, pp. 37-37, 2005.
[12] 黃凱祥, "應用於魔物館Beacon微定位技術:以國立台灣歷史博物館為例," 博物館與文化, no. 15, pp. 5-29, 2018.
[13] 藍介洲, "Beacon 系統應用於視障者室內與戶外定向與引導效用之使用者經驗研究," 身心障礙研究, vol. 16, no. 3-4, 2018.
[14] K. Nikhil , I. Sai Pavan Kalyan ,Jetty Sagar ,M. Sai Rohit , and M. Nesasudha, "Li-FI Smart Indoor Navigation System for Visually Impaired People," 2019 2nd International Conference on Signal Processing and Communication (ICSPC), pp. 187-192, 2019.
[15] B. Li, JP Muñoz, X. Rong, Q. Chen, J. Xiao, Y. Tian, A. Arditi ,and M. Yousuf,, "Vision-Based Mobile Indoor Assistive," IEEE Transactions on Mobile Computing, vol. 18, no. 3, pp. 702-714, 2019.
[16] A. Hub, J. Diepstraten ,and T.Ertl, "Augmented Indoor Modeling for Navigation Support for the Blind," in Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility, 2004.
[17] B. And, S. Baglio, and V. Marletta, "Intelligent sensing solutions for AAL," Sensors, no. 162, pp. 321-324, 2004.
[18] J. Xiao, S. Joseph, X. Zhang, B. Li, X. Li ,and J. Zhang, "An Assistive Navigation Framework for the Visually Impaired," IEEE Transactions on Human-Machine Systems Issue5, vol. 45, no. 5, pp. 635-640, 2015.
[19] J. Redmon and A. Farhadi, "You only look once: Unified, real-time object detection," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788, 2016.
[20] J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," CoRRabx/1612.08242, 2016.
[21] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXivpreprint arXiv:1804.02767, 2018.
[22] X. Li, W. Wang, W. Hou, R.-Z. Liu, T. Lu ,and J. Yang, "Shape Robust Text Detection with Progressive Scale Expansion Network," Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 9336-9345, 2019.
[23] "NVIDA JETSON AGX XAVIER," NVIDIA, [Online]. Available: https://www.nvidia.com/zh-tw/autonomous-machines/jetson-agx-xavier/. [Accessed 6 2020].
[24] "STEREOLABS," [Online]. Available: https://www.stereolabs.com/developers/. [Accessed 6 2020].
[25] liuheng92, "tensorflow_PSENet," [Online]. Available: https://github.com/liuheng92/tensorflow_PSENet. [Accessed 6 9 2019].

指導教授

蘇木春(Mu-Chun Su)

審核日期

2020-8-18

推文