| 摘要: | 本論文旨在建立一個自動識別蔬菜和水果種類並自動計價的無人電子秤系統。本系統採用深度學習中目標檢測(object detection neural network)與圖像分類(image classification neural network)兩種網路相結合的方式來達成目標。當顧客把欲購買之蔬果，無論是否放置於塑膠袋中，置於電子秤上，則該電子秤可自動快速準確地辨識出蔬果類別。根據類別結果對應單價自動計算總價並將單價、重量、總價等資訊顯示在本系統開發之結帳網站中。顧客可使用結帳網站之列印明細功能列印賬單。如此免去顧客在賣場等待過磅時間且節省賣場人工成本，同時達到省時、省錢、無人接觸的目的。
本系統採用Raspberry Pi 4為移動設備端並利用Evdev監測電子秤的重量變化。一個Raspberry Pi 4可以支持多個電子秤同時使用。當重量不為0kg且重量保持不變時，移動設備端控制相機拍照並傳輸圖像與重量到主機辨識端。主機辨識端採用桌上型電腦並利用深度學習Efficient Det的目標檢測網路濾除複雜的背景與塑膠袋之影響，首先取得蔬果位置資訊並截取出多張乾淨的蔬果圖。再使用Efficient Net，ResNet，Mobile Net和Denese Net四個深度學習圖像分類網路分別對多張蔬果圖進行辨識。系統會以各個網路為單位對多張蔬果圖的辨識結果做第一次多數投票法，選取出現次數最多的結果作為該網路的辨識結果。以各個網路在測試資料集的準確率高低對其進行加權多數投票法，得到最終蔬果的結果。為提升辨識速度，本系統採取了平行運算與多線程(multi-threading)的方式，改善因投票機制導致運算時間成本增加的問題。此外為提高辨識準確率，採取Focal Loss Function和SAM(Sharpness-Aware Minimization)提升網路的性能。Focal Loss Function用來解決訓練資料量不平衡，SAM用來提升模型的泛化性。在包含多元變化情況的測試資料集中，本系統總共可辨識40種蔬果，總辨識準確率達96.9%。
;The main purpose of this thesis is to develop a self-service electronic weighing system for recognizing and pricing fruits and vegetables automatically. The object detection neural network and image classification neural network with deep learning are combined to build the system in order to achieve the desired goal. When customers put the fruits and vegetables, no matter inside plastic bags or not, on the electronic scale, their categories can be recognized quickly and accurately as well. The unit price, weight and the total price of fruits or vegetables will be shown on the designed screen and the total price will be calculated. Then customers can print the bill. In this way, the electronic scale can reduce the customer’s waiting time for weighting and save the labor cost for the store, so that the purpose of saving time, saving money and self-service can be reached simultaneously.
This system uses Raspberry Pi 4 as a mobile device to monitor the weight change of the electronic scale. A Raspberry Pi 4 can support multiple electronic scales. The device takes a photo and sends it to the computer when the weight is fixed and not zero. Firstly, the computer uses an object detection neural network Efficient Det to filter out background and plastic bags based on the location information and crop several clean images. Then, we use four neural network models Efficient Net, ResNet, Mobile Net and Denese Net, respectively, to recognize the category of the fruits and vegetables. At last, we make the majority voting for the results from four models to get the final category result. Furthermore, the system adopts parallel and multi-threading to speed up the recognition process. On the other hand, to improve the recognition accuracy, the Focal Loss Function and SAM (Sharpness-Aware Minimization) are used to improve the system performance. The Focal Loss Function is used to solve the imbalance of the training data set and SAM is used to improve the robustness of the system. Based on performing lots of experiments, the proposed system can recognize 40 kinds of fruits and vegetables with 96.9% recognition accuracy. |
