| dc.description.abstract | The goal of this thesis is to establish a recognizing system of NTD (New Taiwan Dollars) banknotes for helping the visually impaired to shop. We combine deep learning neural networks and traditional image processing algorithms to achieve the goal. We designed an APP in which when the mobile phone takes the photos of several banknotes whether they are overlapped or not. The number, denominations and relative positions of the banknotes can be accurately recognized and transmitted by the blind’s smart phone. Therefore, using this APP, the blind can pay the money to shop and check the correctness of the return change from the clerk.
This technology uses the structure of the deep learning neural network Yolact (You Only Look At CoefficienTs) to identify the Chinese characters, digital features, and reflective digital features for each kind of currency banknotes. Then two traditional image processing methods MBR (minimum bounding rectangle) and ORB (Oriented FAST and Rotated BRIEF) algorithms to deal with these features. MBR is used to find the direction of features, the size and relative position of feature boxes. The ORB is used to solve the situation of some broken features, and doing feature positioning. Based on the above two processes, we can find the denomination and quantity of banknotes. Finally, we find the optimal regression lines to fit all features so that the relative positions of the banknotes are obtained. Therefore, the denominations, the relative position and the quantity of banknotes are recognized correctly.
The server of this system is built on the local computer, and the user terminal is designed in the Android mobile phone APP. As long as we use the smartphone and the original built-in camera. The denomination, quantity and relative position of the banknote can be quickly determined. The correct recognition accuracy is about 85% in our experiment with Sony Xperia Z4 smartphone in various situations such as single banknotes and multiple banknotes. | en_US |