基於深度學習之虛擬吉他音樂演奏系統設計;The design of virtual guitar music performance system based on deep learning

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/83947

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/83947

Title:	基於深度學習之虛擬吉他音樂演奏系統設計;The design of virtual guitar music performance system based on deep learning
Authors:	周恒瑋;Zhou, Heng-Wei
Contributors:	資訊工程學系
Keywords:	人機互動介面;深度學習;虛擬樂器;手部偵測;Human–Computer Interaction;Deep Learning;Virtual Instrument;Hand Detection
Date:	2020-07-20
Issue Date:	2020-09-02 17:44:31 (UTC+8)
Publisher:	國立中央大學
Abstract:	人機互動之研究專注於研發使用者與電腦之溝通介面，而如何讓電腦藉由觀察與追蹤辨識演算法之設計來理解輔助裝置（諸如色彩／深度攝影機、智慧手環、傳感手套）之行為數據，為研究學者很重視的議題。近年來有越來越多體驗裝置之產品（如：虛擬頭盔、智慧眼鏡等）被廣泛應用於體感遊戲或是其他應用中，其中包含音樂演奏表演或是藝術表演，目的為讓玩家能夠在無實體硬體設備下（如：鋼琴、吉他等）以更友善且便利之方式演奏樂器，且讓觀眾能有更好的新科技視覺體驗。先前人機互動之研究多以人體骨架之行為理解為主，適用於大動作之行為辨別，近年來各學者以機器學習與深度學習做為行為辨識主要之策略，其提升之準確率證明了機器學習與深度學習之可靠性。為了能夠辨識細部的指尖指法，本論文提出一個以吉他為範例之吉他表演系統，該系統不僅能辨識及他左手之各和絃手勢，且能辨別右手之撥弦演奏行為。經由實驗證明，本系統不僅能被應用在吉他演奏表演系統，亦能應用在其他樂器演奏之表演系統（如：大提琴、小提琴、烏克麗麗等）。除此之外，本論文提出一個手勢辨識之評量機制，能夠用來評估模型與實時演奏之可靠性。 ;Human-computer interaction (HCI) focused on developing the interfaces between users and computers in computer. Many researchers observe the ways in which user interact with computers and design tracking or recognizing mechanism that let computer realized the input command from user’s behavior by auxiliary sensors (e.g. camera, wisdom bracelet, sensing gloves). More and more experiential device (e.g. virtual reality headset, smart glasses) are widely used for users with somatosensory games but they are also used in other applications, including music or artistic performances, the purpose is to allow users to more conveniently control commands on a small number of devices and give audiences a new visual experience. In the past, the researchers mainly focused on the observation and analysis of human skeleton movements by using the design of special algorithms, the computer can understand the limb behavior based on the human skeleton. In our studies. In recent years, more and more machine learning and deep learning approach are used in HCI related researches to prove their reliability. In order to be able to recognize the detailed fingering behavior of the finger, this thesis proposed a guitar playing system, which use deep learning as strategy to recognize the finger gesture between all guitar in left-hand, and picking behavior in right-hand. Also, a verification method for discriminating accuracy was proposed in this thesis, which can be used to prove the reliability of the guitar performance system. Experimental results prove that our system based on deep learning approach can effectively identify the fingering behavior, and also the performance system can be used for other musical instruments (e.g. cello, violin or ukulele).
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	382	View/Open

社群 sharing

Loading...