即時手部追蹤之虛擬樂器演奏系統

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：16

、訪客IP：3.133.12.172

姓名

許木炘(Mu-hsen Hsu) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

即時手部追蹤之虛擬樂器演奏系統
(Designing Virtual Instruments by Real-Time Finger Tracking System)

相關論文

★ 基於edX線上討論板社交關係之分組機制	★ 利用Kinect建置3D視覺化之Facebook互動系統
★ 利用 Kinect建置智慧型教室之評量系統	★ 基於行動裝置應用之智慧型都會區路徑規劃機制
★ 基於分析關鍵動量相關性之動態紋理轉換	★ 基於保護影像中直線結構的細縫裁減系統
★ 建基於開放式網路社群學習環境之社群推薦機制	★ 英語作為外語的互動式情境學習環境之系統設計
★ 基於膚色保存之情感色彩轉換機制	★ 一個用於虛擬鍵盤之手勢識別框架
★ 分數冪次型灰色生成預測模型誤差分析暨電腦工具箱之研發	★ 使用慣性傳感器構建即時人體骨架動作
★ 基於多台攝影機即時三維建模	★ 基於互補度與社群網路分析於基因演算法之分組機制
★ 基於類神經網路之即時虛擬樂器演奏系統	★ 即時手部追蹤系統以虛擬大提琴為例

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

人機互動漸漸成為電腦科學及資訊工程相關研究中的一個重要領域，並且讓人們可以使用各種方式與電腦或設備進行溝通。同時，許多研究者也希望從中發展出更多新的研究議題。Kinect是由微軟公司發行的一個3D感測裝置，使用者可以透過Kinect得到的環境深度資料應用在各種開發和研究。近年來，許多的開發者函式庫陸續的誕生，如微軟官方的 Kinect SDK或PrimeSense公司發布的OpenNI, 這些函式庫提供了包括人體骨架追蹤和深度資料分析的相關技術。然而，這些技術只僅限於人體大節點的追蹤如:手、頭、膝蓋..等。如果需要更細部的偵測如手指，這些開放式函式庫就無法達成，然而 Leap Motion和Senz3d感測器可以達成手指的追蹤，並且提供了相關的函式庫，讓使用者開發自己的應用程式。雖然這些新一代的感測器有相當程度的精確度，並且提供了手指追蹤的函式，但是它有操作距離的限制，使用者必須在固定的位置操作，另外，它所提供的手指追蹤函式庫如果要應用在虛擬樂器的演奏上效率是比較不足的。在這篇論文中，作者設計了一個基於Kinect感測器的手指追蹤演算法，該系統可即時的分析深度資料並且追蹤到使用者的手指。系統可以應用在各個領域或軟體，我們將該系統設計為虛擬樂器的控制器，讓使用者可以透過自身的手勢彈奏樂器。

摘要(英)

Human Computer Interaction is becoming a major component in computer science related fields allowing humans to communicate with machines in very simple ways exploring new dimensions of research. Kinect, the 3D sensing device introduced by Microsoft which allows developers to use depth data for research. Recently, there are several mature libraries such as OpenNI and Kinect SDK which provide functions of Human skeleton tracking. Therefore, users may find that these applications only work with big joints of human body. When it comes to the tracking of fingers, devices such as Leap Motion or Senz3D will be mentioned because it provide APIs (Application Programming Interface) and games, which users can communicate with the computer by their fingers. Although these new devices provides functions of finger tracking, the operating distance is short and users need to sit in a fixed position. On the other hand, the efficiency of current tracking method is hard to be used for Virtual Instrument. Here, in this paper, authors’ experimental efforts on detailed hand tracking including fingers based on Microsoft Kinect. Analyzing the depth information captured by Kinect in real time, users can apply the result in different scopes such as device controller and Virtual Instruments.

關鍵字(中)

★ Kinect
★ Leap Motion
★ 手指追蹤
★ 虛擬樂器
★ OpenNI
★ Kinect SDK

關鍵字(英)

★ Kinect
★ Leap Motion
★ Finger Tracking
★ Virtual Instrument
★ OpenNI
★ Kinect SDK

論文目次

摘要 i
Abstract ii
Acknowledgements iii
Contents iv
List of Figures vi
List of Tables ix
Chapter 1. Introduction 1
1.1 Background 1
1.2 Motivation 3
1.3 Thesis Organization 3
Chapter 2. Related Works 4
2.1 Kinect applications 4
2.2 Hand Gesture Recognition and Fingertips Tracking 6
2.3 MIDI controller 10
2.4 The analysis of existing methods 18
Chapter 3. Proposed method 20
3.1 The work flow of system 20
3.2 Environment 21
3.3 Region of Interest 22
3.4 Center point of hand 26
3.5 Finger region distribution 28
3.6 Finger Tracking 34
3.7 Finger Tapping 36
3.8 Hand is open or not 40
3.9 Robust Method 40
Chapter 4. Applications 43
4.1 Virtual Guitar 43
4.2 Virtual Piano 47
4.3 MIDI 48
Chapter 5. Experimental Results and Discussions 52
5.1 Environment 52
5.2 Experiment 53
5.3 Comparison 61
Chapter 6. Conclusions and Future Works 63
6.1 Conclusions 63
6.2 Future Works 66
References 70

參考文獻

[1] J.L. Raheja, A. Chaudhary, and K. Singal, “Tracking of Fingertips and Centers of Palm Using KINECT”, Computational Intelligence, Modelling and Simulation (CIMSiM), 2011, pp. 248-252.
[2] Zhou Ren, Jingjing Meng, Junsong Yuan, and Zhengyou Zhang, “Robust hand gesture recognition with Kinect sensor”, MM ′11 Proceedings of the 19th ACM international conference on Multimedia, 2011, pp. 759-760.
[3] Yi Li, “Hand Gesture Recognition Using Kinect”, IEEE 3rd International Conference on Software Engineering and Service Science (ICSESS), 2012.
[4] Iason Oikonomidis, Nikolaos Kyriazis and Antonis A. Argyros, “Efficient model-based 3D tracking of hand articulations using Kinect”, in Proceedings of the 22nd British Machine Vision Conference (BMVC) , 2011, University of Dundee, UK, Aug. 29-Sep. 1.
[5] Gabrielle Odowichuk, Shawn Trail, Peter Driessen, Wendy Nie, and Wyatt Page. "Sensor fusion: Towards a fully expressive 3d music control interface." In Communications, Computers and Signal Processing (PacRim), 2011 IEEE Pacific Rim Conference on, pp. 836-841 , 2011.
[6] Marcella Mandanici, and Sylviane Sapir. "DISEMBODIED VOICES: A KINECT VIRTUAL CHOIR CONDUCTOR." Proceedings of the 9th Sound and Music Computing Conference, Copenhagen, Denmark, pp. 271-276 ,2012.
[7] Ying Qin. "A Study of Wii/Kinect Controller as Musical Controllers.", Music Technology Area of Schulich Music School, McGill University Mu-Hsen Hsu, Kumara W.G.C.W., and
[8] Shawn Trail, Michael Dean, Tiago F. Tavares, Gabrielle Odowichuk, Peter Driessen, W. Andrew Schloss, and George Tzanetakis. "Non-invasive sensing and gesture control for pitched percussion hyper-instruments using the Kinect.", The international Conference on New Interfaces for Musical Expression, 2012
[9] Sertan Sentürk, Sang Won Lee, Avinash Sastry, Anosh Daruwalla, and Gil Weinberg. "Crossole: A Gestural Interface for Composition, Improvisation and Performance using Kinect." The international Conference on New Interfaces for Musical Expression, 2012
[10] Jacob Wilschrey, Cristian Rusu, Ivan Mercado, Rodolfo Inostroza, and Cristhy Jiménez. "Virtual Drums Based on Natural Interaction.", The international Conference on Intelligent Networking and Collaborative Systems (INCoS), 2012, pp. 652-655.
[11] Sina Shamshiri, "A Kinect Air Guitar." (2012), A thesis in the Department of Computer Science, University of Sheffield.
[12] Timothy K. Shih, “Spider King: Virtual Musical Instruments based on Microsoft Kinect”, The 6th IEEE International Conference on Ubi-Media Computing, 2013, Aizu-Wakamatsu, Japan, November 2-4.
[13] Charles V. Stewart, ”Robust Parameter Estimation in Computer Vision”, Society for Industrial and Applied Mathematics, 1999, Vol. 41, No. 3, pp. 513–537.

指導教授

施國琛(Timothy k. Shih)

審核日期

2014-7-10

推文