基於AI技術之蔬果辨識計價電子秤

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：112

、訪客IP：18.224.68.104

姓名

董吉峰(Ji-Feng Dong) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

基於AI技術之蔬果辨識計價電子秤

相關論文

★ 直接甲醇燃料電池混合供電系統之控制研究	★ 利用折射率檢測法在水耕植物之水質檢測研究
★ DSP主控之模型車自動導控系統	★ 旋轉式倒單擺動作控制之再設計
★ 高速公路上下匝道燈號之模糊控制決策	★ 模糊集合之模糊度探討
★ 雙質量彈簧連結系統運動控制性能之再改良	★ 桌上曲棍球之影像視覺系統
★ 桌上曲棍球之機器人攻防控制	★ 模型直昇機姿態控制
★ 模糊控制系統的穩定性分析及設計	★ 門禁監控即時辨識系統
★ 桌上曲棍球：人與機械手對打	★ 麻將牌辨識系統
★ 相關誤差神經網路之應用於輻射量測植被和土壤含水量	★ 三節式機器人之站立控制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文旨在建立一個自動識別蔬菜和水果種類並自動計價的無人電子秤系統。本系統採用深度學習中目標檢測(object detection neural network)與圖像分類(image classification neural network)兩種網路相結合的方式來達成目標。當顧客把欲購買之蔬果，無論是否放置於塑膠袋中，置於電子秤上，則該電子秤可自動快速準確地辨識出蔬果類別。根據類別結果對應單價自動計算總價並將單價、重量、總價等資訊顯示在本系統開發之結帳網站中。顧客可使用結帳網站之列印明細功能列印賬單。如此免去顧客在賣場等待過磅時間且節省賣場人工成本，同時達到省時、省錢、無人接觸的目的。
本系統採用Raspberry Pi 4為移動設備端並利用Evdev監測電子秤的重量變化。一個Raspberry Pi 4可以支持多個電子秤同時使用。當重量不為0kg且重量保持不變時，移動設備端控制相機拍照並傳輸圖像與重量到主機辨識端。主機辨識端採用桌上型電腦並利用深度學習Efficient Det的目標檢測網路濾除複雜的背景與塑膠袋之影響，首先取得蔬果位置資訊並截取出多張乾淨的蔬果圖。再使用Efficient Net，ResNet，Mobile Net和Denese Net四個深度學習圖像分類網路分別對多張蔬果圖進行辨識。系統會以各個網路為單位對多張蔬果圖的辨識結果做第一次多數投票法，選取出現次數最多的結果作為該網路的辨識結果。以各個網路在測試資料集的準確率高低對其進行加權多數投票法，得到最終蔬果的結果。為提升辨識速度，本系統採取了平行運算與多線程(multi-threading)的方式，改善因投票機制導致運算時間成本增加的問題。此外為提高辨識準確率，採取Focal Loss Function和SAM(Sharpness-Aware Minimization)提升網路的性能。Focal Loss Function用來解決訓練資料量不平衡，SAM用來提升模型的泛化性。在包含多元變化情況的測試資料集中，本系統總共可辨識40種蔬果，總辨識準確率達96.9%。

摘要(英)

The main purpose of this thesis is to develop a self-service electronic weighing system for recognizing and pricing fruits and vegetables automatically. The object detection neural network and image classification neural network with deep learning are combined to build the system in order to achieve the desired goal. When customers put the fruits and vegetables, no matter inside plastic bags or not, on the electronic scale, their categories can be recognized quickly and accurately as well. The unit price, weight and the total price of fruits or vegetables will be shown on the designed screen and the total price will be calculated. Then customers can print the bill. In this way, the electronic scale can reduce the customer’s waiting time for weighting and save the labor cost for the store, so that the purpose of saving time, saving money and self-service can be reached simultaneously.
This system uses Raspberry Pi 4 as a mobile device to monitor the weight change of the electronic scale. A Raspberry Pi 4 can support multiple electronic scales. The device takes a photo and sends it to the computer when the weight is fixed and not zero. Firstly, the computer uses an object detection neural network Efficient Det to filter out background and plastic bags based on the location information and crop several clean images. Then, we use four neural network models Efficient Net, ResNet, Mobile Net and Denese Net, respectively, to recognize the category of the fruits and vegetables. At last, we make the majority voting for the results from four models to get the final category result. Furthermore, the system adopts parallel and multi-threading to speed up the recognition process. On the other hand, to improve the recognition accuracy, the Focal Loss Function and SAM (Sharpness-Aware Minimization) are used to improve the system performance. The Focal Loss Function is used to solve the imbalance of the training data set and SAM is used to improve the robustness of the system. Based on performing lots of experiments, the proposed system can recognize 40 kinds of fruits and vegetables with 96.9% recognition accuracy.

關鍵字(中)

★ 深度學習
★ 目標檢測
★ 圖像分類
★ 蔬果辨識

關鍵字(英)

★ deep learning
★ object detection
★ image classification
★ fruits and vegetables recognition

論文目次

摘要 i
Abstract ii
致謝 iv
目錄 v
圖目錄 vii
表目錄 ix
第一章緒論 1
1.1 研究動機與背景 1
1.2 文獻回顧 2
1.3 論文目標 6
1.4 論文架構 7
第二章系統架構與軟硬體介紹 8
2.1 系統架構與流程圖 8
2.2 硬體介紹 9
2.2.1 電子秤型號及功能 9
2.2.2 相機型號及功能 10
2.2.3 列印機型號及功能 10
2.2.4 Raspberry Pi型號及功能 11
2.2.5 桌上型電腦 11
2.3 軟體系統介紹 12
第三章目標檢測網路 13
3.1 網路架構 13
3.2 訓練資料 17
第四章蔬果識別演算法 23
4.1 圖像分類網路架構 23
4.2 資料收集 27
4.3 網路優化 30
第五章外接設備及結帳網站 37
5.1 電子秤傳輸並讀取重量數據 37
5.1.1 電子秤傳輸原理 37
5.1.2 讀取重量到程式中 38
5.2 連接Raspberry Pi 4與主機電腦 40
5.3 結帳網站開發 42
5.3.1 開發環境 42
5.3.2 介面展示 43
第六章操作使用說明 46
6.1 外接設備安裝說明 46
6.2 顧客結帳說明 49
第七章實驗結果 52
7.1 目標檢測網路結果 52
7.2 圖像分類網路結果 54
7.3 模型集成結果 55
7.3.1 無塑膠袋之單個蔬果辨識結果 55
7.3.2 無塑膠袋之多個蔬果辨識結果 57
7.3.3 有塑膠袋之單個蔬果辨識結果 58
7.3.4 有塑膠袋之多個蔬果辨識結果 59
第八章結論與未來展望 60
8.1 結論 60
8.2 未來展望 61
參考文獻 62

參考文獻

[1] C. Huang, Y. Wang, X. Li, L. Ren, J. Zhao, Y. Hu, et al, “Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China,” Lancet, 2020, vol. 395, no. 10223, pp. 497–506.
[2] "COVID-19-Taiwan,"[Online]. Available: https://sites.google.com/cdc.gov.tw/2019ncov/taiwan. [Accessed: June, 2021].
[3] "農產品批發市場交易行情站," [Online]. Available: https://amis.afa.gov.tw/m_fruit/FruitChartMarketTransVolumeScale.aspx. [Accessed: June, 2021].
[4] G. Zeng, "Fruit and vegetables classification system using image saliency and convolutional neural network," in Proceedings of 2017 IEEE 3rd Information Technology and Mechatronics Engineering Conference (ITOEC), 3-5 Oct.,2017: IEEE, pp. 613-617.
[5] S. Jana and R. Parekh, "Shape-based fruit recognition and classification," in Proceedings of International Conference on Computational Intelligence, Communications, and Business Analytics, 26 September 2017: Springer, pp. 184-196.
[6] H. Mureşan and M. Oltean, "Fruit recognition from images using deep learning," Acta Universitatis Sapientiae, Informatica,, 2018, vol. 10, no. 1, pp. 26-42, 4 Jan..
[7] "Fruit and Vegetable Detection and Feature Extraction Using Instance Segmentation-Part-1,"[Online].Available: https://medium.com/codalyze/fruit-and-vegetable-detection-and-feature-extraction-using-instance-segmentation-part-1-178cc73122cb. [Accessed: June, 2021].
[8] "水果識別-測試, " [Online]. Available: https://zhuanlan.zhihu.com/p/49591788. [Accessed: June, 2021].
[9] 邹谜，伍世虔與王欣, "一種用於機器人水果採摘的快速識別方法," 碩士, 冶金裝備及其控制, 武漢科技大學, 2019.
[10] 廉小亲, 成开元與安飒, 吴叶兰, 关文洋, "基於深度學習和遷移學習的水果圖像分類," 計算機與信息工程學院, 北京工商大學, 食品安全大數據技術, 2019.
[11] I. B. Mustaffa and S. F. B. M. Khairul, "Identification of fruit size and maturity through fruit images using OpenCV-Python and Rasberry Pi," 2017 International Conference on Robotics, Automation and Sciences (ICORAS), Melaka, Malaysia, 2017, pp.1-3.
[12] E. Rachmawati, I. Supriana and M. L. Khodra, "Toward a new approach in fruit recognition using hybrid RGBD features and fruit hierarchy property," 2017 4th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), Yogyakarta, Indonesia, 2017, pp. 1-6.
[13] Y. He, F. Pan, B. Wang, Z. Teng and J. Wu, "Transfer Learning Based Fruits Image Segmentation for Fruit-Picking Robots," 2020 IEEE 3rd International Conference on Computer and Communication Engineering Technology(CCET), Beijing, China, 2020, pp.71-75.
[14] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[15] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv preprint arXiv:2004.10934, 2020.
[16] S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, 6 Jan.,2016, vol. 39, no. 6, pp. 1137-1149.
[17] M. Tan, R. Pang and Q. V. Le, " EfficientDet: Scalable and Efficient Object Detection," arXiv preprint arXiv:1911.09070, 2019.
[18] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 10 Dec., 2015, pp. 770-778.
[19] A. G, Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, and H. Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications, " arXiv preprint arXiv:1704.04861, 2017.
[20] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, "Densely Connected Convolutional Networks," arXiv preprint arXiv:1608.06993, 2018.
[21] S. Gao, M.-M. Cheng, K. Zhao, X.-Y. Zhang, M.-H. Yang, and P. H. Torr, "Res2net: A new multi-scale backbone architecture," In Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence, 1 Sep.,2019.
[22] M. Tan and Q. V. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," arXiv preprint arXiv:1905.11946, 2019.
[23] P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur, "Sharpness-Aware Minimization for Efficiently Improving Generalization," arXiv preprint arXiv:2010.01412, 2020.
[24] T. Y. Lin, P. Goyal, R. Girshick, K. He, & P. Dollár, "Focal loss for dense object detection," In Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980-2988.
[25] "Woolworths‘s mind-blowing artificial intelligence speeding up check-outs," [Online]. Available: https://www.news.com.au/lifestyle/food/eat/woolworths-hidden-camera-at-checkouts-makes-shopping-faster/news-story/62b2f85fc468b59bc75a326708a96aea. [Accessed: June, 2021].
[26] "Intelligent scales provide comfortable self-service experience," [Online]. Available: https://www.digisystem.com/solutions/SOL00065/. [Accessed: June, 2021].
[27] "python 實時檢測鍵盤輸入函數," [Online]. Available: https://blog.csdn.net/qq_40930675/article/details/84667762. [Accessed: June, 2021].
[28] "Linux中使用evdev捕獲鍵盤(遙控器)輸入," [Online]. Available: https://blog.csdn.net/ManWZD/article/details/103376521. [Accessed: June, 2021].
[29] "鈺恆JWM-30K 外貿出口電子秤110V寬電壓," [Online]. Available: https://world.taobao.com/item/595919691836.htm. [Accessed: June, 2021].
[30] "Logitech C270 HD 網絡攝影機," [Online]. Available: https://www.logitech.com/zh-tw/products/webcams/c270-hd-webcam.960-000626.html. [Accessed: June, 2021].
[31] "GP3120 203DPI 熱敏桌面USB 接口條形碼物流單標籤打印機," [Online]. Available: https://chinese.alibaba.com/product-detail/gp3120-203dpi-thermal-desktop-usb-interface-barcode-logistics-single-label-printer-62501757650.html. [Accessed: June, 2021].
[32] "Raspberry Pi 4:處理器大升級，性能直逼入門電腦," [Online]. Available: https://www.techapple.com/archives/30761. [Accessed: June, 2021].
[33] T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, "Feature Pyramid Networks for Object Detection," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 936-944.
[34] S. Liu, L. Qi, H. F. Qin, J. P. Shi, and J. Y. Jia, "Path Aggregation Network for Instance Segmentation," arXiv preprint arXiv:1803.01534, 2018.
[35] "LabelImg," [Online]. Available: https://github.com/tzutalin/labelImg. [Accessed: June, 2021].
[36] "Data_Augmentation_Zoo_for_Object_Detection," [Online]. Available: https://github.com/zzlpointcloud/Data_Augmentation_Zoo_for_Object_Detection. [Accessed: June, 2021].
[37] B. Zoph, and Q. V. Le, "Neural Architecture Search with Reinforcement Learning," arXiv preprint arXiv:1611.01578, 2017.
[38] F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions," arXiv preprint arXiv:1610 02357, 2017.
[39] J. Hu, L. Shen, S. Albanie, G. Sun, and E. H. Wu, "Squeeze-and-Excitation Networks," arXiv preprint arXiv:1610.02357, 2019.
[40] H. Chockler, E. Farchi, B. Godlin and S. Novikov, "Cross-Entropy Based Testing," Formal Methods in Computer Aided Design (FMCAD′07), 2017, pp. 101-108.

指導教授

王文俊(Wen-June Wang)

審核日期

2021-7-6

推文