| dc.description.abstract | Human-computer interaction (HCI) focused on developing the interfaces between users and computers in computer. Many researchers observe the ways in which user interact with computers and design tracking or recognizing mechanism that let computer realized the input command from user’s behavior by auxiliary sensors (e.g. camera, wisdom bracelet, sensing gloves). More and more experiential device (e.g. virtual reality headset, smart glasses) are widely used for users with somatosensory games but they are also used in other applications, including music or artistic performances, the purpose is to allow users to more conveniently control commands on a small number of devices and give audiences a new visual experience.
In the past, the researchers mainly focused on the observation and analysis of human skeleton movements by using the design of special algorithms, the computer can understand the limb behavior based on the human skeleton. In our studies. In recent years, more and more machine learning and deep learning approach are used in HCI related researches to prove their reliability. In order to be able to recognize the detailed fingering behavior of the finger, this thesis proposed a guitar playing system, which use deep learning as strategy to recognize the finger gesture between all guitar in left-hand, and picking behavior in right-hand. Also, a verification method for discriminating accuracy was proposed in this thesis, which can be used to prove the reliability of the guitar performance system. Experimental results prove that our system based on deep learning approach can effectively identify the fingering behavior, and also the performance system can be used for other musical instruments (e.g. cello, violin or ukulele). | en_US |