本論文旨在設計一個具回收分類的機械手臂抓取系統，藉由導入機器視覺與深度學習網路，經由攝影機辨識畫面中未知回收物的種類，並控制六軸機械手臂將畫面中的物品夾取到指定的分類，指定分類內容為塑膠、玻璃、紙容器、金屬容器、保麗龍共五類。 本論文之研究項目敘述如下，透過裝置在機械手臂末端的RGB-D攝影機所輸出的影像完成以下項目：(1)使用深度學習實例分割(instance segmentation)網路分割與辨識物品；(2)利用影像處理技術改善塑膠以及玻璃物品因透明造成深度不完整的原始深度圖像；(3)利用實例分割結果找出各物件的輪廓並計算各物品的面積及中心；(4)透過攝影機內部參數使用透視投影法將深度轉為點雲；(5)將點雲資訊輸入深度學習夾取檢測(grasp detection)網路並輸出抓取參數；(6)將夾取參數轉換為實際夾取位置以及角度並進行中心篩選。另外針對六軸機械手臂夾取目標物品的過程，完成以下項目：(1)使用正向運動學設立不同初始點以及不同分類的終點；(2)建立虛擬環境防止機械手臂在運動過程中與真實環境的障礙物發生碰撞；(3)利用逆向運動學推算機械手臂末端中心到達抓取點時各軸旋轉角度； (4)設定關節角度限制以避免姿態轉換造成機械手臂末端大幅度位移；(5)將位於攝影機坐標系下抓取檢測網路的輸出進行座標轉換用來控制機械手臂到達夾取點。綜合以上技術，可以在符合影像大小與機械手臂結構限制下，運用逆向運動學控制機械手臂夾取目標物品。 本研究在Linux環境下使用機器人作業系統(Robot Operating System, ROS)開發軟體系統，透過ROS分散式的架構與點對點網路，將所有資訊收集在一起並進行資料傳遞並整合，實現軟體硬體協同的設計。 ;This thesis aims to design a robotic arm-grabbing system for recyclables classification. The unknown recyclables in the image will be identified by RGB-D camera and deep learning network, and the six-axis robotic arm will be controlled to grab the recyclables to the specified boxes. The considered recyclables are plastic, glass, paper container, metal container, and styrofoam. This thesis focuses on utilizing the output images from an RGB-D camera mounted at the end of a robotic arm to achieve several objectives. They are as follows. (1) A deep learning instance segmentation network is employed to segment and classify objects in the captured images effectively. (2) Image processing techniques are applied to enhance the depth information of transparent things, such as plastic and glass, which may have incomplete depth measurements. (3) The instance segmentation results are utilized to extract the contours of each identified object and then we can enable to calculate their respective areas and centers. (4) Using the camera′s internal parameters, the depth information is transformed into a point cloud. (5) Input the point cloud information into the deep learning grasp detection (grasp detection) network and output the grasping parameters. (6) Convert the grasping parameters into the actual grasping position and angle and then perform center screening. In addition, for the process of the six-axis robot arm picking up the target object, the following tasks are completed. (1) Using forward kinematics to set up different initial points and end points for the robot arm operation. (2) Establishing a virtual environment to prevent collision happening during robot movement. (3) Using inverse kinematics to calculate the rotation angle of each axis when the center of the end of the robot arm reaches the grab point. (4) Setting the joint angle constraints to avoid a major shift at the end of the robot. (5) Calculating the transformation matrix of grasp detection network under the camera frame for the robot arm to reach the grasping point. After the above tasks completed, it is possible to use inverse kinematics to control the robotic arm to grab the target object under the limitation of vision range and the constraints of the mechanism. This thesis uses the Robot Operating System (Robot Operating System, ROS) to develop the software system in the Linux environment. All information is collected, transmitted, and integrated through the distributed architecture of ROS and the peer-to-peer network to achieve software and hardware collaboration.
