基於多個Kinect的虛擬球體樂器

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：29

、訪客IP：3.134.104.173

姓名

呂名峰(Ming-Fong Lu) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於多個Kinect的虛擬球體樂器
(3D Sphere Virtual Instrument with multiple Kinect)

相關論文

★ 基於edX線上討論板社交關係之分組機制	★ 利用Kinect建置3D視覺化之Facebook互動系統
★ 利用 Kinect建置智慧型教室之評量系統	★ 基於行動裝置應用之智慧型都會區路徑規劃機制
★ 基於分析關鍵動量相關性之動態紋理轉換	★ 基於保護影像中直線結構的細縫裁減系統
★ 建基於開放式網路社群學習環境之社群推薦機制	★ 英語作為外語的互動式情境學習環境之系統設計
★ 基於膚色保存之情感色彩轉換機制	★ 一個用於虛擬鍵盤之手勢識別框架
★ 分數冪次型灰色生成預測模型誤差分析暨電腦工具箱之研發	★ 使用慣性傳感器構建即時人體骨架動作
★ 基於多台攝影機即時三維建模	★ 基於互補度與社群網路分析於基因演算法之分組機制
★ 即時手部追蹤之虛擬樂器演奏系統	★ 基於類神經網路之即時虛擬樂器演奏系統

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

人機互動的領域在現代被廣泛地運用，在這篇論文中我們設計一個虛擬樂器，主要是使用微軟公司的KINECT感應器，能夠偵測人體的骨架來操作音樂，KINECT本身有限制人體面對KINECT的角度而導致側面無法被偵測，本論文使用骨架當作點雲，並使用三台KINECT來對齊骨架而達到側面也可以被偵測。在樂器中我們使用一顆球作為介面，介面中的球有12個表面並可以彈奏單音或和弦，為了達到12的表面都可以被彈奏，我們使用ICP演算法來對齊三台KINECT讓KINECT側面可以被偵測到，來達到使用者可以盡情的彈奏樂器而不會受到正面彈奏的限制。在介面的部份我們使用OpenGL來做為繪圖的基礎，論文中會介紹基礎的三維觀念和OpenGL基礎，並理解如何創造一顆虛擬球體且放入人體的骨架當作樂器，並介紹MIDI相關的知識與如何使用MIDI來彈奏音樂，最後實驗的部分會顯示出骨架對其的準確度與顯示骨架對齊的圖示。

摘要(英)

Human Computer Interaction is wildly used in our life, in this paper we design a virtual instrument by the Microsoft Kinect sensor. Kinect sensor can detect human body by representing a skeleton, and the program use the skeleton to control the music signal. Kinect has human body angle limits, if user’s side face to Kinect then Kinect cannot detect skeleton, this paper uses skeletons as point cloud and calibrate three Kinect to achieve side face detection. In our instrument we use a sphere for interface, inside the sphere has twelve blocks and the user can play a single note or chords. In order to play all of the blocks, we use the ICP algorithm to calibrate three Kinect and the users can the play instrument without side face limits. At program interface we use OpenGL for rendering the objects, and we will introduce the basic three dimensional concepts and OpenGL concept, explains how to put the skeleton inside the sphere and introduce MIDI concept to use MIDI for playing music. Finally the experiment result shows the accuracy of the skeleton in calibration and show the status of skeleton in figures.

關鍵字(中)

★ 人機互動
★ Kinect
★ MIDI
★ 虛擬樂器
★ OpenGL
★ 感應器校正
★ 點雲
★ 點雲對齊

關鍵字(英)

★ HCI
★ Kinect
★ MIDI
★ Virtual Instument
★ OpenGL
★ Kinect calibration
★ PointCloud
★ ICP

論文目次

摘要 vi
Abstract vii
Acknowledgements viii
Contents ix
List of Figures xi
List of Tables xv
Chapter 1. Introduction 1
Chapter 2. Related work 3
2.1 3D Concept 5
1. 3D effect 6
2. Coordinate systems 6
3. Cartesian coordinates 7
4. 3D Cartesian Coordinates 7
5. Projection 8
6. Orthographic projection 9
7. Perspective projection 9
8. Coordinate Clipping 9
9. Viewport 10
2.2 Kinect 11
1. Detect user 13
2. Kinect SDK 13
3. Kinect skeleton 16
2.3 OPENGL 17
1. Vertex basic 17
2. Pipeline 18
3. OpenGL API 18
4. GLUT 19
5. OPENGL TRIANGLE EXAMPLE 22
6. Opengl translate 23
7. Rotated 24
2.4 MIDI 25
2.5 ICP algorithm 31
1. ICP Principle 32
Chapter 3. Proposed method 34
3.1 Creation of virtual space 34
3.2 Two Kinect Calibration 43
1. The old method 43
2. The new method 47
Chapter 4. System architecture 56
Chapter 5. Experiment result and Analysis 61
Experiment in 30 degrees with 6 samples 67
Chapter 6. Conclusion and Future works 72
1. Future work 72
2. Conclusion 73
References 74

參考文獻

[1] D. Jung, S. Laing, M. H. Jensen, P. Hunkin, A. Löf, and N. Tims, Requirements on dance-driven 3-D camera interaction: a collaboration between dance, graphic design and computer science. In Proceedings of the 12th Annual Conference of the New Zealand Chapter of the ACM Special Interest Group on Computer-Human Interaction (pp. 25-32). ACM. (2011, July).
[2] M. Tang, "Recognizing hand gestures with microsoft’s kinect." Palo Alto: Department of Electrical Engineering of Stanford University:[sn] (2011).
[3] W. Tao, Y. Qiao, and B. Lee. "Kinect Skeleton Coordinate Calibration for Remote Physical Training." MMEDIA 2014, The Sixth International Conferences on Advances in Multimedia. 2014.
[4] S. Kaenchan, P. Mongkolnam, B. Watanapa, and S. Sathienpong, "Automatic multiple kinect cameras setting for simple walking posture analysis." Computer Science and Engineering Conference (ICSEC), 2013 International. IEEE, 2013.
[5] M. H. Hsu, W. G. C. W. Kumara, T. K. Shih, and Z. Cheng, “Spider King: Virtual musical instruments based on microsoft Kinect,” in 2013 International Joint Conference on Awareness Science and Technology and Ubi-Media Computing (iCAST-UMEDIA), Ulaanbaatar, 2013, pp. 707-713.
[6] S. Sentürk, S. W. LEE, A. Sastry, A. Daruwalla, and G. Weinberg "Crossole: A gestural interface for composition, improvisation and performance using kinect." Proc. NIME. Vol. 2012. 2012.
[7] MIDI MANUFACTURES ASSOCIATION. MIDI [Online].Available: https://www.midi.org/index.php
[8] Microsoft. KinectSDK. [Online].Available:https://msdn.microsoft.com/en-us/library/hh855347.aspx

[9] Solarian.(2013,05,22). OpenGL101: Matrices - projection, view, model.[Online].Available:https://solarianprogrammer.com/2013/05/22/opengl-101-matrices-projection-view-model/
[10] Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing (IVC), 10(3):145-155, 1992. 2.3, 3.5
[11] P. Besl, and N. McKay, "A Method for Registration of 3-D Shapes", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no.2, pp.239-256, Feb., 1992
[12] R. S. Wright, and B. Lipchak, (2004). OpenGL superbible.
[13] Sorkine, O., 2007. Least-squares rigid motion using svd, [Online].Available http://igl.ethz.ch/projects/ARAP/svd_rot.pdf.
[14] F. Colas, 2011. Iterative Closest Point Algorithm [Online]. Available: http://www.asl.ethz.ch/education/master/info-process-rob/ICP.pdf
[15] K. Low. Linear least-squares optimization for point-to-plane icp surface registration. Technical report, Technical Report TR04-004, Department of Computer Science, University of North Carolina at Chapel Hill, 2004.
[16] J. Wilm, 2013. Iterative Closest Point matlab code [Online]. Available:
http://www.mathworks.com/matlabcentral/fileexchange/27804-iterative-closest-point

[17] H. Kjer, and J. Wilm. Evaluation of surface registration algorithms for PET motion correction. Master′s thesis, Technical University of Denmark, 2010. 7.

[18] dfrec 2010.12.07 OpenGL入门教程 [Online]. Available:
http://wenku.baidu.com/view/8096a02bcfc789eb172dc892.html

[19] Y. H. Hsiao, An Application of MIDI in Guitar′′s Scoring. Master’s thesis, Tamkang University, 1999.

[20] J. wolfe, 2005. Note names, MIDI numbers and frequencies [Online]. Available:
http://newt.phys.unsw.edu.au/jw/notes.html

[21] S. Kaenchan, P. Mongkolnam, B. Watanapa, and S. Sathienpong, Automatic multiple Kinect cameras setting for simple walking posture analysis Computer Science and Engineering Conference (ICSEC), 2013 International, Nakorn Pathom, pp.245-249, 4-6 Sept. 2013.

[22] A. Saeed Matlab code icp.m [Online]. Available:
http://staffhome.ecm.uwa.edu.au/~00053650/code.html

指導教授

施國琛(Timothy K. Shih)

審核日期

2015-7-24

推文