| dc.description.abstract | In this era, the progress of science and technology is much faster than people think. The interaction between humans and machines is not limited to the interaction between the keyboard mouse and the screen.
In the past, we believe virtual interfaces and acoustic lighting effects that can be only seen in the movie will no longer depend on the imagination of actors.
Instead, technologies nowadays can provide users with immersive experiences through virtual reality technology.
Different cameras and sensors like virtual reality headset or glove bring users a whole new experience.
The advancement of science and technology comes from people′s needs.
In today′s society, entertainment has become an indispensable part of life.
Kinect is a depth camera developed by Microsoft that can track the body′s skeleton by capturing depth information.
By recognizing the position of human joints, users can use the movement of the limbs to interact with the machine, and we have realized the design of virtual percussion instruments through these depth information.
Due to hardware limitations, the Kinect camera′s classifier cannot determine the masked limbs or the subtle and fast movements.
For example, in the process of playing percussion instrument, we often use the subtle movements of our fingers that can not be captured by camera to change the position of tapping .
Therefore, we hope to solve these problems by installing six-axis inertial sensors on mallet to obtain acceleration and angular acceleration data
The rapid development of machine learning in recent years has made it impossible to ignore, and it has made considerable achievements in different fields.
In our thesis, we will collect triaxial accelerations and angular accelerations of hand movements by designing virtual instruments and use these signals as the basis dataset for gesture recognition.
When the user play a virtual musical instrument, the user can input a gesture by writing in the air, and execute a command such as changing the pitch or tone according to the recognition result.
In our opinion, the combination of multi-axis signal and machine learning not only compensates for the inborn defects of the camera, but also expands people′s imagination for human-computer interaction. | en_US |