本論文目標是建立一個方便視障人士使用的新台幣鈔票辨識系統。使用深度學習的架構結合影像處理的演算法，本系統使用深度學習的架構結合影像處理的演算法達成目標。當使用者以手機拍攝多張面額不限之鈔票且無論重疊與否，都可以準確辨識鈔票的面額、數量和相對位置，並用語音告知使用者。如此視障人士就可以用手機輕易辨識鈔票付錢購物或是檢查店員找錢金額的正確性了。 此技術利用深度學習Yolact (You Only Look At CoefficienTs)的架構，辨識出每種不同幣值鈔票的國字特徵、數字特徵、反光數字特徵，再利用MBR (minimum bounding rectangle) 及 ORB (Oriented FAST and Rotated BRIEF)兩種影像處理的演算法對各個特徵進行處理。MBR用來找到特徵的方向，特徵框的大小及相對位置。ORB用來解決一些特徵破碎的情況，並做特徵定位。利用上兩個方法所得的結果去做分析，求出鈔票的面額及數量。最後把所有的特徵，去做排列組合，並利用線性回歸找出所有組合的回歸線，找到最短距離的最佳解。利用最佳解將鈔票的相對位置計算出來。 本系統的伺服器建立在本地端電腦上，使用者端則是設計在Android手機APP內，只要利用智慧型手機和原有的內建鏡頭，就可以快速的判別鈔票的面額、數量和相對位置。我們以智慧型手機Sony Xperia Z4來測試，在單一鈔票和多張鈔票等各種情況下，平均有85%的準確率。 ;The goal of this thesis is to establish a recognizing system of NTD (New Taiwan Dollars) banknotes for helping the visually impaired to shop. We combine deep learning neural networks and traditional image processing algorithms to achieve the goal. We designed an APP in which when the mobile phone takes the photos of several banknotes whether they are overlapped or not. The number, denominations and relative positions of the banknotes can be accurately recognized and transmitted by the blind’s smart phone. Therefore, using this APP, the blind can pay the money to shop and check the correctness of the return change from the clerk. This technology uses the structure of the deep learning neural network Yolact (You Only Look At CoefficienTs) to identify the Chinese characters, digital features, and reflective digital features for each kind of currency banknotes. Then two traditional image processing methods MBR (minimum bounding rectangle) and ORB (Oriented FAST and Rotated BRIEF) algorithms to deal with these features. MBR is used to find the direction of features, the size and relative position of feature boxes. The ORB is used to solve the situation of some broken features, and doing feature positioning. Based on the above two processes, we can find the denomination and quantity of banknotes. Finally, we find the optimal regression lines to fit all features so that the relative positions of the banknotes are obtained. Therefore, the denominations, the relative position and the quantity of banknotes are recognized correctly. The server of this system is built on the local computer, and the user terminal is designed in the Android mobile phone APP. As long as we use the smartphone and the original built-in camera. The denomination, quantity and relative position of the banknote can be quickly determined. The correct recognition accuracy is about 85% in our experiment with Sony Xperia Z4 smartphone in various situations such as single banknotes and multiple banknotes.
