| dc.description.abstract | This study integrates reinforcement learning with a robotic arm to enable ultrasound scanning on abdominal surfaces. A depth camera was first used for 3D modeling to construct
virtual abdominal models. These virtual objects were utilized to train the robotic arm to perform ultrasound scans on uncertain surfaces. During the training process, inverse kinematics control was implemented based on the Denavit-Hartenberg (DH) parameter table of the UR5e robotic arm, ensuring precise control of the virtual robotic arm′s movements. The reinforcement learning method employed the Proximal Policy Optimization (PPO) network, where the virtual robotic arm autonomously learned through a reward mechanism. In this framework, the Actor selects actions, while the Critic evaluates the value of these actions and
adjusts the Actor′s strategy through a reward mechanism. This approach enables the robotic arm to continuously learn and improve its strategies in uncertain environments, achieving optimal ultrasound scanning performance.After completing model training, the Intel Realsense
D435 depth camera was used to capture depth images of actual surface objects, which were then imported into PyBullet for modeling. This allowed the replacement of the random virtual
objects used during training, facilitating real-world applications. On the object models generated in PyBullet, the trained reinforcement learning model was applied to determine joint angles for the virtual robotic arm′s scanning path. These joint angles were transmitted to the
physical UR5e robotic arm via the Real-Time Data Exchange (RTDE) communication protocol, enabling it to replicate the actions of the virtual robotic arm and perform ultrasound
scanning.For the control of the actual robotic arm, the built-in force sensing system of the UR5e robotic arm was used to maintain the applied force within a safe range, preventing excessive force at the tool center point (TCP). This ensured safety and accuracy during the scanning process. The force control mechanism effectively prevented damage to the scanned objects due to overexertion while enhancing the accuracy and reliability of the scanning
viii data.Experimental results demonstrated that the trained robotic arm could perform precise ultrasound scanning on various abdominal surfaces, even with uncertainties. Additionally, the modeling error was only 0.00000079 m2, and the applied force during scanning was
consistently maintained between 9N and 10N, with an average of 9.65N. Regarding cosine distance, the mean cosine distance was 0.0017, close to 0, indicating a high alignment of the ultrasound probe with the abdominal surface′s normal vectors. | en_US |