單目視覺於智慧倉儲的棧板辨識

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：13

、訪客IP：18.117.107.90

姓名

林俐秀(Li-Xiu Lin) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

單目視覺於智慧倉儲的棧板辨識
(Monocular Vision to Pallet Recognition in Smart Warehousing)

相關論文

★ 整合GRAFCET虛擬機器的智慧型控制器開發平台	★ 分散式工業電子看板網路系統設計與實作
★ 設計與實作一個基於雙攝影機視覺系統的雙點觸控螢幕	★ 智慧型機器人的嵌入式計算平台
★ 一個即時移動物偵測與追蹤的嵌入式系統	★ 一個固態硬碟的多處理器架構與分散式控制演算法
★ 基於立體視覺手勢辨識的人機互動系統	★ 整合仿生智慧行為控制的機器人系統晶片設計
★ 嵌入式無線影像感測網路的設計與實作	★ 以雙核心處理器為基礎之車牌辨識系統
★ 基於立體視覺的連續三維手勢辨識	★ 微型、超低功耗無線感測網路控制器設計與硬體實作
★ 串流影像之即時人臉偵測、追蹤與辨識─嵌入式系統設計	★ 一個快速立體視覺系統的嵌入式硬體設計
★ 即時連續影像接合系統設計與實作	★ 基於雙核心平台的嵌入式步態辨識系統

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-7-14以後開放)

摘要(中)

在智慧倉儲業，叉車機器人使用了各種感測器來偵測環境和測距，用以移動、避障
和辨識和存取倉儲棧板。傳統使用 3D 攝影機以獲取精準的棧板位置和距離資訊，但其
成本高昂、計算速度慢，並且空間解析度不高。本研究提出了一個基於單目視覺影像來
進行棧板物件辨識和距離預測的方法，稱為 MVPRP(Monocular Vision for Pallet
recognition and positioning)，此方法使用了 YOLACT 網路模型來從事即時 2D 棧板辨識
和定位，並藉由 ResNet 模型估測 2D 影像所缺乏的棧板距離資訊，使其能在低硬體設備
成本、即時運算的情況下解決倉儲自動化中叉車機器人對於棧板偵測和距離估測的問題。

摘要(英)

In the smart warehousing industry, forklift robots utilize various sensors for environment
detection, ranging, and the recognition and retrieval of warehouse pallets. Traditional
approaches employ 3D cameras to obtain accurate pallet positions and distance information,
but they are costly, computationally slow, and have limited spatial resolution. This study
proposes a method called MVPRP (Monocular Vision for Pallet Recognition and Positioning)
that relies on monocular visual imagery for pallet object recognition and distance estimation.
The approach utilizes the YOLACT network model for real-time 2D pallet recognition and
localization, and leverages the ResNet model to estimate the missing distance information from
the 2D images. This enables the solution to address pallet detection and distance estimation
challenges in warehouse automation for forklift robots, while maintaining low hardware costs
and real-time computation.

關鍵字(中)

★ 棧板辨識
★ 距離估測

關鍵字(英)

論文目次

摘要 I
Abstract II
誌謝 III
圖目錄 VII
表目錄 X
第一章、緒論 1
1.1 研究背景 1
1.2 研究目的 3
1.3 論文架構 3
第二章、文獻回顧 4
2.1 倉儲自動化叉車機器人系統 4
2.1.1 3D物件辨識 4
2.1.2 棧板辨識和定位演算法 7
2.2 2D物件辨識 8
2.2.1 2D物件偵測 8
2.2.2 2D物件分割 10
2.3 距離偵測 14
第三章、棧板辨識與定位設計 16
3.1 單目視覺棧板辨識與定位方法(MVPRP) 16
3.1.1 棧板辨識 17
3.1.2 棧板距離估測 19
3.2 MVPRP系統架構設計 23
3.2.1 IDEF0階層式模組化設計 24
3.2.2 MVPRP階層式系統架構 25
3.2.3 棧板辨識模組 26
3.2.4 距離估測模組 26
3.3 MVPRP系統離散事件建模 27
3.3.1 GRAFCET離散事件建模 27
3.3.2 MVPRP離散事件建模 29
3.3.3 棧板辨識模組離散事件建模 29
3.3.4 棧板提取離散事件建模 30
3.3.5 距離估測模組離散事件建模 31
3.4 MVPRP系統軟體高階合成 33
第四章、系統整合 38
4.1 實驗環境 38
4.1.1 實驗軟硬體環境規格 38
4.1.2 實驗用棧板3D列印 40
4.1.3 棧板辨識資料集 41
4.1.4 棧板距離估測資料集 43
4.2 MVPRP系統整合 45
4.2.1 Server平台測試 46
4.2.2 Nvidia Jetson TX2平台測試 48
第五章、 MVPRP系統實驗 49
5.1 棧板辨識實驗 49
5.1.1 棧板辨識資料集分佈狀態 50
5.1.2 基於廣角資料集之棧板辨識 52
5.1.3 基於超廣角資料集之棧板辨識 55
5.2 棧板距離估測實驗 57
5.2.1 距離估測資料集分佈狀況 57
5.2.2 基於廣角資料集之距離估測 60
5.2.3 基於廣角資料集之物件位置估測 62
5.2.4 基於超廣角資料集之棧板距離估測系統 65
5.3 討論 67
第六章、結論 68
6.1 結論 68
6.2 未來展望 69
參考文獻 70

參考文獻

[1] Y. Li, X. Chen, and G. Ding, "Pallet Localization Techniques of Forklift Robot: A Review of Recent Progress," J Robot Mech Eng, vol. 1, pp. 1-7, 2021.
[2] D. Adolfsson, M. Magnusson, A. Alhashimi, A. J. Lilienthal, and H. Andreasson, "Lidar-level localization with radar? the cfear approach to accurate, fast, and robust large-scale radar odometry in diverse environments," IEEE Transactions on robotics, vol. 39, no. 2, pp. 1476-1495, 2022.
[3] S. Wang, X. Chen, G. Ding, Y. Li, W. Xu, Q. Zhao, Y. Gong, and Q. Song, "A lightweight localization strategy for LiDAR-guided autonomous robots with artificial landmarks," Sensors, vol. 21, no. 13, p. 4479, 2021.
[4] R. Iinuma, Y. Hori, H. Onoyama, T. Fukao, and Y. Kubo, "Pallet Detection and Estimation for Fork Insertion with RGB-D Camera," in 2021 IEEE International Conference on Mechatronics and Automation (ICMA), pp. 854-859, 2021.
[5] F. Jia, Z. Tao, and F. Wang, "Pallet detection based on Halcon for warehouse robots," in 2021 IEEE International Conference on Artificial Intelligence and Industrial Design (AIID), pp. 401-404, 2021.
[6] J. Xiao, H. Lu, L. Zhang, and J. Zhang, "Pallet recognition and localization using an rgb-d camera," International Journal of Advanced Robotic Systems, vol. 14, no. 6, p. 1729881417737799, 2017.
[7] Y.-Y. Li, X.-H. Chen, G.-Y. Ding, S. Wang, W.-C. Xu, B.-B. Sun, and Q. Song, "Pallet detection and localization with RGB image and depth data using deep learning techniques," in 2021 6th International Conference on Automation, Control and Robotics Engineering (CACRE), pp. 306-310, 2021.
[8] Z. Yang and L. Wang, "Learning relationships for multi-view 3D object recognition," in Proceedings of the IEEE/CVF international conference on computer vision, pp. 7505-7514, 2019.
[9] Z. Ren, I. Misra, A. G. Schwing, and R. Girdhar, "3d spatial recognition without spatially labeled 3d," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13204-13213, 2021.
[10] Y. He, H. Yu, X. Liu, Z. Yang, W. Sun, Y. Wang, Q. Fu, Y. Zou, and A. Mian, "Deep learning based 3D segmentation: A survey," arXiv preprint arXiv:2103.05423, 2021.
[11] F. Ashiq, M. Asif, M. B. Ahmad, S. Zafar, K. Masood, T. Mahmood, M. T. Mahmood, and I. H. Lee, "CNN-based object recognition and tracking system to assist visually impaired people," IEEE Access, vol. 10, pp. 14819-14834, 2022.
[12] S. Minaee, Y. Y. Boykov, F. Porikli, A. J. Plaza, N. Kehtarnavaz, and D. Terzopoulos, "Image segmentation using deep learning: A survey," IEEE transactions on pattern analysis and machine intelligence, 2021.
[13] A. Mousavian, D. Anguelov, J. Flynn, and J. Kosecka, "3d bounding box estimation using deep learning and geometry," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 7074-7082, 2017.
[14] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, "Scannet: Richly-annotated 3d reconstructions of indoor scenes," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5828-5839, 2017.
[15] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, "Pointnet: Deep learning on point sets for 3d classification and segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017.
[16] X. Shi, Q. Ye, X. Chen, C. Chen, Z. Chen, and T.-K. Kim, "Geometry-based distance decomposition for monocular 3d object detection," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15172-15181, 2021.
[17] J. Shashirangana, H. Padmasiri, D. Meedeniya, and C. Perera, "Automated license plate recognition: a survey on methods and techniques," IEEE Access, vol. 9, pp. 11203-11225, 2020.
[18] Q. Guan, Y. Chen, Z. Wei, A. A. Heidari, H. Hu, X.-H. Yang, J. Zheng, Q. Zhou, H. Chen, and F. Chen, "Medical image augmentation for lesion detection using a texture-constrained multichannel progressive GAN," Computers in Biology and Medicine, vol. 145, p. 105444, 2022.
[19] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788, 2016.
[20] Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, "Yolox: Exceeding yolo series in 2021," arXiv preprint arXiv:2107.08430, 2021.
[21] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors," arXiv preprint arXiv:2207.02696, 2022.
[22] K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, pp. 2961-2969, 2017.
[23] X. Qin, Z. Zhang, C. Huang, M. Dehghan, O. R. Zaiane, and M. Jagersand, "U2-Net: Going deeper with nested U-structure for salient object detection," Pattern recognition, vol. 106, p. 107404, 2020.
[24] D. Bolya, C. Zhou, F. Xiao, and Y. J. Lee, "Yolact: Real-time instance segmentation," in Proceedings of the IEEE/CVF international conference on computer vision, pp. 9157-9166, 2019.
[25] C. Mok, I. Baek, Y. S. Cho, Y. Kim, and S. B. Kim, "Pallet Recognition with Multi-Task Learning for Automated Guided Vehicles," Applied Sciences, vol. 11, no. 24, p. 11808, 2021.
[26] S. Oswal and D. Saravanakumar, "Line following robots on factory floors: Significance and Simulation study using CoppeliaSim," in IOP Conference Series: Materials Science and Engineering, vol. 1012, no. 1, p. 012008, 2021.
[27] I. S. Mohamed, A. Capitanelli, F. Mastrogiovanni, S. Rovetta, and R. Zaccaria, "A 2D laser rangefinder scans dataset of standard EUR pallets," Data in brief, vol. 24, p. 103837, 2019.
[28] J. Zhao, B. Li, X. Wei, H. Lu, E. Lü, and X. Zhou, "Recognition and Location Algorithm for Pallets in Warehouses Using RGB-D Sensor," Applied Sciences, vol. 12, no. 20, p. 10331, 2022.
[29] G. H. Beckman, D. Polyzois, and Y.-J. Cha, "Deep learning-based automatic volumetric damage quantification using depth camera," Automation in Construction, vol. 99, pp. 114-124, 2019.
[30] A. Singandhupe and H. M. La, "A review of slam techniques and security in autonomous driving," in 2019 third IEEE international conference on robotic computing (IRC), pp. 602-607, 2019.
[31] Y. Li, L. Ma, Z. Zhong, F. Liu, M. A. Chapman, D. Cao, and J. Li, "Deep learning for lidar point clouds in autonomous driving: A review," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 8, pp. 3412-3432, 2020.
[32] M. Liu, X. Zhang, and H. Su, "Meshing point clouds with predicted intrinsic-extrinsic ratio guidance," in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16, pp. 68-84, 2020.
[33] C. He, R. Li, S. Li, and L. Zhang, "Voxel set transformer: A set-to-set approach to 3d object detection from point clouds," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8417-8427, 2022.
[34] J. Deng, W. Zhou, Y. Zhang, and H. Li, "From multi-view to hollow-3D: Hallucinated hollow-3D R-CNN for 3D object detection," IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 12, pp. 4722-4734, 2021.
[35] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, "Pointnet++: Deep hierarchical feature learning on point sets in a metric space," Advances in neural information processing systems, vol. 30, 2017.
[36] S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," Advances in neural information processing systems, vol. 28, 2015.
[37] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "Ssd: Single shot multibox detector," in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pp. 21-37, 2016.
[38] P. Ram and K. Sinha, "Revisiting kd-tree for nearest neighbor search," in Proceedings of the 25th acm sigkdd international conference on knowledge discovery & data mining, pp. 1378-1388, 2019.
[39] J. Priesnitz, C. Rathgeb, N. Buchmann, C. Busch, and M. Margraf, "An overview of touchless 2D fingerprint recognition," EURASIP Journal on Image and Video Processing, vol. 2021, no. 1, pp. 1-28, 2021.
[40] I. Adjabi, A. Ouahabi, A. Benzaoui, and A. Taleb-Ahmed, "Past, present, and future of face recognition: A review," Electronics, vol. 9, no. 8, p. 1188, 2020.
[41] R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, pp. 1440-1448, 2015.
[42] P. Adarsh, P. Rathi, and M. Kumar, "YOLO v3-Tiny: Object Detection and Recognition using one stage improved model," in 2020 6th international conference on advanced computing and communication systems (ICACCS), pp. 687-694, 2020.
[43] D. Ristić-Durrant, M. Franke, and K. Michels, "A review of vision-based on-board obstacle detection and distance estimation in railways," Sensors, vol. 21, no. 10, p. 3452, 2021.
[44] S. Salagrama, H. H. Kumar, R. Nikitha, G. Prasanna, K. Sharma, and S. Awasthi, "Real time social distance detection using Deep Learning," in 2022 International Conference on Computational Intelligence and Sustainable Engineering Solutions (CISES), pp. 541-544, 2022.
[45] T. Roddick, A. Kendall, and R. Cipolla, "Orthographic feature transform for monocular 3d object detection," arXiv preprint arXiv:1811.08188, 2018.
[46] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
[47] C.-H. Chen, M.-Y. Lin, and X.-C. Guo, "High-level modeling and synthesis of smart sensor networks for Industrial Internet of Things," Computers & Electrical Engineering, vol. 61, pp. 48-66, 2017.
[48] R. Vashistha, P. Kumar, A. K. Dangi, N. Sharma, D. Chhabra, and P. Shukla, "Quest for cardiovascular interventions: precise modeling and 3D printing of heart valves," Journal of biological engineering, vol. 13, no. 1, pp. 1-12, 2019.

指導教授

陳慶瀚(Qing-Han Chen)

審核日期

2023-7-24

推文