| dc.description.abstract | The purpose of this thesis is to control the six-axis robotic arm to keep a fixed distance from the human face and aim at the center of the forehead with the aids of computer vision and deep learning. Furthermore, when the human face has a little movement, the robot can also follow it within the limited range. This tracking system can be applied to interactive tasks between humans and robots. In this thesis, it is considered that the arm holding a mobile phone or tablet maintains the above distance and direction to face the user’s face. Even if the face moves or rotates a little, it can still track the face with the set distance and direction, so that the user can watch the mobile phone or tablet without holding them. To achieve this, the robotic arm needs to detect the human face, recognize the facial features, and calculate the related positions between the face and the terminal of robotic arm. Finally, the robotic arm will implement the face tracking by using inverse kinematics.
In face detection and facial features recognition, both are achieved by deep learning technique, after that we also (1) obtain the position of facial features points in the image and (2) calculate the related positions between the facial features and the camera. In addition, the movement control of the robotic arm also needs the following process. (1) building a virtual environment; (2) calculating the transfer matrix of the model; (3) finding the coordinates of the target point and using inverse kinematics to control the robotic arm moving to the target point. This study also added a function in the mobile phone to fine-tune the pose of the robotic arm. The fine tune operation can be implemented by manual input or voice input. The mobile phone can also display the images taken by the current camera on the robot arm simultaneously. This technology can also be applied to the automatic forehead temperature measurement during the current epidemic situation (2020 COVID-19).
In this thesis, Robot Operating System (ROS) is used in the Linux operating system and completed the transmission and integration of information between laptop, robotic arm, camera, and mobile phone.
Keywords: Face detection, Facial landmark, 6 DOF robotic arm, Kinematics, Robot operating system, Coordinate transformation, Face tracking. | en_US |