基於深度學習與影像處理技術之單眼視覺六軸機械手臂控制

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：65

、訪客IP：13.59.46.85

姓名

張華延(Hun-Yen Chang) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

基於深度學習與影像處理技術之單眼視覺六軸機械手臂控制

相關論文

★ 直接甲醇燃料電池混合供電系統之控制研究	★ 利用折射率檢測法在水耕植物之水質檢測研究
★ DSP主控之模型車自動導控系統	★ 旋轉式倒單擺動作控制之再設計
★ 高速公路上下匝道燈號之模糊控制決策	★ 模糊集合之模糊度探討
★ 雙質量彈簧連結系統運動控制性能之再改良	★ 桌上曲棍球之影像視覺系統
★ 桌上曲棍球之機器人攻防控制	★ 模型直昇機姿態控制
★ 模糊控制系統的穩定性分析及設計	★ 門禁監控即時辨識系統
★ 桌上曲棍球：人與機械手對打	★ 麻將牌辨識系統
★ 相關誤差神經網路之應用於輻射量測植被和土壤含水量	★ 三節式機器人之站立控制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文主要目的為藉著視覺系統的導入以逆向運動學控制六軸機械手臂移動並且夾取五種不同的物品。經由影像偵測與辨識目標物品以及計算目標物品與機械手臂之間的相對位置，在符合影像大小以及機械手臂的機構限制下，將目標物品隨意放置皆能完成夾取。
本研究在Linux環境下使用機器人作業系統(Robot Operating System, ROS)開發軟體系統，透過ROS分散式的架構與點對點網路，將所有資訊收集在一起進行資料傳遞並整合NVIDIA Jetson TX2、六軸機械手臂、工業相機以及夾爪，實現軟體硬體協同的設計。
本論文之研究項目敘述如下，透過裝置在機械手臂末端的工業相機所輸出的單眼視覺影像完成以下三點：(1)使用深度學習偵測與辨識物品；(2)利用影像處理技術改善深度學習技術輸出的物品邊框(Bounding Box)；(3)透過相機針孔模型計算出目標物品與相機之間的相對位置。另外針對六軸機械手臂夾取目標物品的過程，我們完成以下項目：(1)使用正向運動學求出兩個不同座標系之間的相對位置以及相對角度；(2)設立空間中特定點利用逆向運動學求算工具端中心到達該點時各軸旋轉角度；(3)建置虛擬環境防止機械手臂在運動過程中與障礙物發生碰撞；(4)設定關節角度限制以避免姿態轉換造成機械手臂末端大幅度位移;(5)運用路徑限制避免運動過程中與目標物品發生碰撞；(6)利用路徑規劃建立初始點與目標點中之間的中繼點。綜合以上技術，可以在符合影像大小與機械手臂結構限制下，運用逆向運動學控制機械手臂夾取目標物品。

摘要(英)

The main purpose of this paper is to control six degrees of freedom (6DOF) robot to achieve a pick-and-place application for five different objects. The relative position between the object and the robot is calculated via using vision to detect and identify the objects, in addition, the objects are randomly placed inside the mechanical limit of the robot and satisfy the image size of vision. By receiving the information from vision, the robot can successfully pick and place the object.
Robot operating system (ROS) is used to develop a software system under Linux environment in this study. The NVIDIA Jetson TX2, the robot, the industrial camera and the gripper are integrated by ROS distributed architecture and peer-to-peer network, and all information and data collected can be transferred to them as well. Therefore, the collaborative design is used to realize the integrated software and hardware.
The most important point is that we use machine vision to detect, identify the target objects and calculate the relative position between each object and the robot arm. We complete the following three steps through the monocular vision from the industrial camera which is mounted on the end of the robot. First, we use deep learning technology to detect and identify objects. Second, we improve the bounding box of deep learning technology result by using Image process technology. Third, we calculate the relative position between the object and the camera by the pinhole camera model. With regard to the robot application, we complete the following tasks. First, the relative position and angle between two different frames are calculated by using forward kinematics. Second, a specific point is set and every joint angle is calculated through inverse kinematics when the robot tool center arrives at the point. Third, we set a virtual environment to prevent collision happening during robot movement. Fourth, we set the joint angle constraints to avoid a major shift at the end of the robot. Fifth, using path constraint to prevent collision between the robot and the target object. Sixth, a series of middle points between the initial point and the target point can be found by using trajectory planning. After the above tasks completed, the robot can implement a randomly pick-and-place application through inverse kinematics under the limitation of vision range and the constraints of the mechanism.

關鍵字(中)

★ 運動學
★ 六軸機械手臂
★ ROS
★ 單眼視覺
★ 深度學習
★ 影像處理
★ 路徑規劃

關鍵字(英)

★ Kinematics
★ 6 DOF robot
★ ROS
★ Monocular vision
★ Deep learning
★ Image processing
★ Trajectory planning

論文目次

摘要 i
Abstract ii
致謝 iv
目錄 v
圖目錄 vii
表目錄 xi
第一章緒論 1
1.1研究背景與動機 1
1.2文獻回顧 2
1.3論文目標 4
1.4論文架構 4
第二章系統架構與硬體介紹 5
2.1系統架構 5
2.2硬體架構 6
第三章物品偵測與辨識以及實驗流程規劃 12
3.1物品偵測與辨識 12
3.1.1改善物品邊框 14
3.1.2場景反光 18
3.1.3目標物品部分顏色與背景顏色相似 20
3.2針孔投影法 22
3.3機器人作業系統的應用 27
3.3.1 ROS簡介 27
3.3.2 Moveit 30
3.3.3 ROS 套件使用 33
3.3.4實驗節點與主題示意圖 39
第四章機械手臂運動學與應用 42
4.1轉移矩陣 43
4.2正向運動學 44
4.3逆向運動學 45
4.4運動學的應用 48
第五章實驗結果 50
5.1工作環境介紹 50
5.2目標物品在書櫃中放置的位置 51
5.3影像偵測與辨識結果 55
5.3.1深度學習成果 55
5.3.2深度學習資訊結合影像處理技術成果 57
5.4夾取應用 58
5.4.1第一次夾取應用 58
5.4.2第二次夾取應用 65
第六章結論與未來展望 68
6.1結論 68
6.2未來展望 68
參考文獻 70

參考文獻

[1] 行政院，"行政院生產力4.0發展方案"，2015年9月。
Available at: http://class.nchu.edu.tw/bulletin/MOE/105_MoE_re_allr.pdf
[2] "交通大學開放式課程-機器人學"，2019年6月。
Available at: http://ocw.nctu.edu.tw/course_detail.php?bgid=8&gid=0&nid=554
[3] M. Zeiler and R. Fergus. "Visualizing and understanding convolutional networks," in Proc. European Conference on Computer Vision, Springer, Sep. 2014, pp. 818-833.
[4] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conference Computer Vision and Pattern Recognition, Columbus, Jun. 2014, pp. 580-587.
[5] R. B. Girshick, "Fast R-CNN," in Proc. International Conference on Computer Vision Pattern Recognition, Santiago, Dec. 2015, pp. 1440-1448.
[6] S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real time object detection with region proposal networks," in Proc. IEEE Transactions on Pattern Analysis Machine Intelligence, vol. 39, pp. 1137-1149, 2017.
[7] J. Redmon, and A. Farhadi, "YOLOv3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[8] D. Bao and P. Wang, "Vehicle distance detection based on monocular vision," in Proc. IEEE International Conference Progress in Informatics and Computing, Shanghai, Dec. 2016, pp. 187-191.
[9] Z. Xu, L. Wang, and J. Wang, "A method for distance measurement of moving objects in a monocular image," in Proc. The 3rd IEEE Conference Signal and Image Process, Shenzhen, July 2018, pp. 245-249.
[10] R. K. Megalingam, V. Shriram, B. Likhith, G. Rajesh and S. Ghanta, "Monocular distance estimation using pinhole camera approximation to avoid vehicle crash and back-over accidents," in Proc. The 10th International Conference on Intelligent Systems and Control, Coimbatore, Jan. 2016, pp. 1-5.
[11] L. Jianguo, L Weidong, G, Li-e, and L. Le, "Detection and localization of underwater targets based on monocular vision," in Proc. The 2nd International Conference on Advanced Robotics and Mechatronics, Hefei, Aug. 2017, pp. 100-105.
[12] X. Li, and Lu Wang, "A monocular distance estimation method used in video sequence," in Proc. International Conference on Information and Automation, Shenyang, Jun. 2012, pp. 390-394
[13] Robot end effector – Wikipedia, June 2019.
Available at : https://en.wikipedia.org/wiki/Robot_end_effector
[14] R. C. Luo, T.-W. Lin, and Y.-H. Tsai, "Analytical inverse kinematic solution for modularized 7-DOF redundant manipulators with offsets at shoulder and wrist," in Proc. IEEE/RSJ International Conference on Intelligent Robots and System, Chicago, Sep. 2014, pp. 516-521.
[15] J. Peng, W. Xu, Z. Wang, and D. Meng, "Analytical inverse kinematics and trajectory planning for a 6DOF grinding robot," in Proc. IEEE International Conference Information and Automation, Yinchuan, Aug. 2013, pp. 834-839.
[16] J.-J. Kim and J.-J. Lee, "Trajectory optimization with particle swarm optimization for manipulator motion planning," in Proc. IEEE Transactions on Industrial Informatics, vol. 11, pp. 620-631, Mar. 2015.
[17] 粒子群最佳化-維基百科，2019年6月。
Available at : https://zh.wikipedia.org/wiki/%E7%B2%92%E5%AD%90%E7%BE%A4%E4%BC%98%E5%8C%96
[18] J. Vannoy and J. Xiao, "Real-time adaptive motion planning (RAMP) of mobile manipulators in dynamic environments with unforeseen changes," in Proc. IEEE Transactions on Robotics, vol. 24, pp. 1199-1212, Oct. 2008.
[19] L. Ruya, et al., "Kinematic control of redundant robots and the motion optimizability measure," in Proc. IEEE Transaction on Systems, Man and Cybernetics, Part B, vol. 31, pp. 155-160, Feb. 2001.
[20] S. Klemm, J. Oberlander, A. Hermann, A. Roennau, T. Schamm, J. M. ¨ Zollner, and R. Dillmann, "RRT*-Connect: Faster, asymptotically optimal motion planning," in Proc. IEEE International Conference on Robotics and Biomimetics, Zhuhai, Dec. 2015, pp. 1670-1677.
[21] M. Mediavilla, J. R. Perán, and L. J.Miguel, "On-line path planning for robot manipulators in dynamic environments," in Proc. European Control Conference, Porto, Sep. 2001, pp. 1167-1173.
[22] L. C. Antonio-Gopar, C. Lopez-Franco, N. Arana-Daniel, E. Gonzalez-Vallejo, and A. Y. Alanis, "Inverse kinematics for a manipulator robot based on differential evolution algorithm," in Proc. IEEE Latin American Conference on Computational Intelligence, Gudalajara, Nov. 2018, pp. 1-5.
[23] R. Storn and K. Price, “Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces,” in Proc. Journal of Global Optimization, vol. 11, no.4, pp. 341-359, Dec. 1997.
[24] Y. Cui, P. Shi, and J. Hua. "Kinematics analysis and simulation of a 6-DOF humanoid robot manipulator," in Proc. International Asia Conference on Informatics in Control, Automation and Robotics, Wuhan, Apr. 2010, pp. 246-249.
[25] Denavit–Hartenberg parameters – Wikipedia, June 2019
Available at : https://en.wikipedia.org/wiki/Denavit%E2%80%93Hartenberg_parameters
[26] J. Xiao, W. Han and A. Wang, "Simulation research of a six degrees of freedom manipulator kinematics based on MATLAB Toolbox," in Proc. International Conference on Advanced Mechatronic Systems, Xiamen, Mar. 2017, pp. 376-380.
[27] J. J. Kuffner Jr and S. M. LaValle, "RRT-Connect: An efficient approach to single-query path planning," in Proc. IEEE International Conference on Robotics and Automation, San Francisco, Aug. 2000, pp. 995-1001.
[28] YOLOv3 tiny – GitHub, June 2019
Available at : https://github.com/pjreddie/darknet/tree/master/cfg
[29] "YOLOv3_tiny修改網路架構教學網站"，2019年6月。
Available at : http://chtseng.wordpress.com/2018/09/01/%E5%BB%BA%E7%AB%8B%E8%87%AA%E5%B7%B1%E7%9A%84yolo%E8%BE%A8%E8%AD%98%E6%A8%A1%E5%9E%8B-%E4%BB%A5%E6%9F%91%E6%A9%98%E8%BE%A8%E8%AD%98%E7%82%BA%E4%BE%8B/
[30] 鍾國亮編者，"影像處理與電腦視覺"，東華書局，2015年出版。
[31] " Moveit官方網站文件檔案觀念說明"，2019年6月。
Available at : https://moveit.ros.org/documentation/concepts/

指導教授

王文俊(Wen-June Wang)

審核日期

2019-7-16

推文