| 摘要: | 本論文的目的為藉由導入機器視覺與深度學習網路，控制六軸機械手臂與人臉保持固定距離且能對準額頭中央，並在有限範圍內主動跟隨人臉移動。此追蹤系統可應用於人與機器人的互動任務，在本論文中則是以考慮該手臂握持手機或平板與人臉保持上述距離與方向，即使人臉有些許移動或轉動，仍然可以追蹤人臉並維持設定好的距離與方向，方便使用者可以不用自己手持也能看手機或平板。為達成此目的，機械手臂端需經由人臉偵測與辨識臉部特徵，計算人臉與機械手臂末端的相對位置，並由逆向運動學完成人臉的追蹤任務。
在人臉偵測與臉部特徵辨識部分，此二功能皆透過深度學習達成，並且完成(1)取得面部特徵在影像中的位置；(2)計算特徵點與攝影機的相對位置。另外機械手臂的運動控制亦需以下程序。(1)建置虛擬環境；(2)計算模型的轉換矩陣；(3)求得目標點的座標，並以逆運動學控制機器手臂到目標點。綜合上述條件，便可在機械手臂自身機構限制內完成追蹤臉部任務。除了影像辨識與機械手臂的控制外，本研究也添加了手機應用程式供使用者微調機械手臂的位置，可透過手動或語音輸入來調整機械手臂進行上下左右偏移，亦可於手機上同步顯示當前機械手臂上的攝影機所拍攝的影像。本研究亦將此技術應用於目前疫情期間(2020 COVID-19)的自動額溫量測，當人靠近機械手臂前，便能偵測該人的臉部位置，並將測溫模組對準額頭為使用者量測額溫，若有發燒將會出現警示，如此便可免除人工量測的負擔。
本研究在Linux環境下使用機器人作業系統(Robot Operating System, 以下皆簡稱ROS)，由於ROS能共享資訊的特點，便可透過節點與節點間訊息的發送與接收，完成系統中不同軟硬體間的資訊溝通。本研究藉由ROS完成了筆記型電腦、六軸機械手臂、攝影機以及手機程式間資訊的傳遞與彙整，實現軟硬體協同應用。
;The purpose of this thesis is to control the six-axis robotic arm to keep a fixed distance from the human face and aim at the center of the forehead with the aids of computer vision and deep learning. Furthermore, when the human face has a little movement, the robot can also follow it within the limited range. This tracking system can be applied to interactive tasks between humans and robots. In this thesis, it is considered that the arm holding a mobile phone or tablet maintains the above distance and direction to face the user’s face. Even if the face moves or rotates a little, it can still track the face with the set distance and direction, so that the user can watch the mobile phone or tablet without holding them. To achieve this, the robotic arm needs to detect the human face, recognize the facial features, and calculate the related positions between the face and the terminal of robotic arm. Finally, the robotic arm will implement the face tracking by using inverse kinematics.
In face detection and facial features recognition, both are achieved by deep learning technique, after that we also (1) obtain the position of facial features points in the image and (2) calculate the related positions between the facial features and the camera. In addition, the movement control of the robotic arm also needs the following process. (1) building a virtual environment; (2) calculating the transfer matrix of the model; (3) finding the coordinates of the target point and using inverse kinematics to control the robotic arm moving to the target point. This study also added a function in the mobile phone to fine-tune the pose of the robotic arm. The fine tune operation can be implemented by manual input or voice input. The mobile phone can also display the images taken by the current camera on the robot arm simultaneously. This technology can also be applied to the automatic forehead temperature measurement during the current epidemic situation (2020 COVID-19).
In this thesis, Robot Operating System (ROS) is used in the Linux operating system and completed the transmission and integration of information between laptop, robotic arm, camera, and mobile phone.
Keywords: Face detection, Facial landmark, 6 DOF robotic arm, Kinematics, Robot operating system, Coordinate transformation, Face tracking. |
