多模態體感動作辨識系統之人機介面研究

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：144

、訪客IP：3.21.248.119

姓名

黃秀珊(HUANG,SIOU-SHAN) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

多模態體感動作辨識系統之人機介面研究
(Multi-modal human–Machine Interaction system for Human Motion Recognition.)

相關論文

★ 虛擬實境搭配腦電、心電以及呼吸器設備在心肺同步呼吸訓練對心跳變異與腦波之訓練應用系統與資料分析	★ 利用分層共現網絡評估發展遲緩兒童的精細運動
★ 太極大師：基於太極拳的注意力訓練遊戲，使用動作辨識及平衡分析進行表現評估	★ 比較XRSPACE MANOVA中手勢和控制器互動模式的用戶體驗
★ 基於舌頭力量和表面肌電圖的吞嚥智能評估系統	★ 基於數據融合模型的機器學習對甲基苯丙胺使用障礙的多生理訊號號分析
★ 在有干擾的虛擬教室環境下大人小孩的行為表現與腦神經反應的異同	★ 使用映射模型和跨資料集遷移式學習的輕量化居家衰弱症訓練系統
★ 心率生理回饋放鬆訓練對於海洛因使用疾患(HUD)生理資訊之影響分析	★ 基於深度學習模型的3D心理旋轉對認知障礙的診斷與評估
★ 評估注意力偵測之穿戴式腦電電極放置有效性	★ 基於骨架步態藉由機器學習進行臨床老化衰落分類
★ 用於注意力不足過動症診斷的可解釋多模態融合模型	★ 基於VR的自閉症兒童多模態訓練系統的改進
★ 基於深度學習的虛擬現實腦震盪檢測與融合方法	★ 設計透過立體互動的虛擬實境教學遊戲系統

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

人機互動、智能生活是現行電腦科學研究領域中十分熱門的議題，而人體動作辨識更是被認為達到這些目標的重要關鍵。而在微軟推出Kinect之後，有越來越多的研究單位選擇使用較為低成本且普及的Kinect來做姿態辨識。本論文透過結合無線感測器改善Kinect 先天性的偵測弱點，針對當部分肢體平貼於地面或有肢體重疊的姿態進行姿態辨識。
本論文將對使用Kinect、無線感測器、以及Kinect結合感測器等三套系統進行比較當使用不同裝置於姿態辨識的差異，其姿態辨識率、任務完成時間、使用者感受等。
使用Kinect搭配OpenNI結合無線感測器所產生的人體骨架的三維座標(x，y，z)值，投影到Unity 3D 遊戲場景中，利用3D骨架呈現人體姿態，而其骨架資訊經過正規化後，使用自我組織特徵映射網路(SOM)進行分群，再與正確樣本進行即時性(real-time)的比對。
本論文提出的Kinect體感技術結合無線感測器模組輔助系統在姿態辨識上有平均94.6%的平均辨識率，明顯比其他兩套系統來的高，可以得出其系統整合了兩種體感技術於姿態辨識的優點。本系統在辨識複雜度高的姿勢動作時，也能有平均94.6%的成功辨識率。

摘要(英)

In this study, we use wireless sensors combined with Kinect to improve the detection weakness of Kinect which is some parts of human bodies flat on the ground or gesture overlapping. In human gesture recognition, we compare Kinect system, sensor system, and Kinect combined sensor system those three systems in four ways: devices recognize difference, gesture recognition rate, task completion time, user’s satisfaction.
Three-dimensional coordinate (x, y, z) values, which is 3D human skeletons’ data using Kinect with OpenNI combining with wireless sensors, projected to Unity 3D game scene and it shows result and compares with the correct sample by using self-organizing map network (SOM) in real time. This study proposes a combination of wireless sensor and Kinect system in human gesture recognition with an average of 94.6% on the average recognition rate.

關鍵字(中)

★ 姿態辨識
★ Kinect
★ 無線感測器
★ 自我組織特徵映射網路

關鍵字(英)

★ human gesture recognition
★ Kinect
★ wireless sensors
★ SOM

論文目次

目錄
摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 viii
第一章緒論 1
1-1 研究動機 1
1-2 研究目的 2
1-3 論文架構 5
第二章相關研究 6
2-1 傳統WEBCAM於姿態辨識上的研究 6
2-2 無線感測器於人體姿態辨識上的研究 7
2-3 KINECT 於人體姿態辨識上的研究 12
2-4 演算法(類神經網路)的相關研究 15
第三章研究方法：系統設計 17
3-1 動作感測方法 17
3-1-1 Kinect體感技術輔助系統(簡稱Kinect系統) 17
3-1-2 無線感測器模組輔助系統(簡稱Sensor系統) 18
3-1-3 Kinect體感技術結合無線感測器模組輔助系統(K+S系統) 23
3-2 動作姿態辨識方法 23
3-2-1自我組織特徵映射圖(SOM)演算法介紹 23
3-2-2 使用SOM來實現動作姿態分群 25
3-3姿態辨識系統 29
3-3-1 系統架構 29
3-3-2 多重表徵使用者介面設計 30
3-3-3 任務設計 31
第四章研究方法：實驗設計 37
4-1 受試者 37
4-2 實驗流程 37
4-3 問卷設計 39
第五章結果與討論 41
5-1姿態辨識演算法之辨識率驗證 41
5-2任務表現：任務完成率 42
5-3任務表現：任務平均完成時間 45
5-4 運動軌跡 46
5-5 問卷結果與分析 47
第六章結論與未來展望 50
第七章參考文獻 51
附錄 57
圖目錄

圖 1 研究架構圖 4
圖 2 星型骨架〔22〕 7
圖 3 9DOF RAZOR IMU (HMC5883L) 8
圖 4 各種天線型態的XBEE模組 (A)晶片型 (B)鞭型天線 (C)PCB印刷型天線 10
圖 5 L. WENFENG 〔25〕提出的七個感測器之實際裝置圖 11
圖 6 跌倒偵測之實際配戴圖〔30〕 12
圖 7 感測器服飾〔31〕 12
圖 8即時辨識系統〔32〕 14
圖 9　KINECT 運作流程圖 18
圖 10　全身感測器裝置圖 19
圖 11 XBEE EXPLORER(正反面) 19
圖 12　無線傳輸流程圖 20
圖 13　無線感測器之組成 20
圖 14　ARDUINO　軟體畫面 21
圖 15　X-CTU軟體畫面 22
圖 16　X-CTU　MODEN CONFIGURATION設定畫面 22
圖 17　KINECT與無線感測器結合之傳輸流程圖 23
圖 18　KINECT 全身關節圖〔44〕 26
圖 19　鄰近區域函數 (A)正方形 (B)六邊形的型式 28
圖 20系統架構圖 30
圖 21　操作介面之介紹 31
圖 22 動作1 32
圖 23 動作2 32
圖 24 動作3 33
圖 25 動作4 33
圖 26 動作5 34
圖 27 動作6 34
圖 28 動作7 35
圖 29 動作8 35
圖 30 動作9 36
圖 31　個別系統實驗流程圖 38
圖 32 動作七之任務成功圖(K+S系統) 39
圖 33 動作七之任務失敗圖(KINECT系統) 39
圖 34　科技接受度模型〔45〕 40
圖 35動作四之下肢運動軌跡(A)KINECT (B)無線感測器 47
圖 36 動作五之下肢運動軌跡(A)KINECT (B)無線感測器 47

表目錄

表 1　SOM參數設定 28
表 2　任務動作之類神經元編號 29
表 3 三種演算法的辨識率 42
表 4 在不同環境下之任務完成率 42
表 5不同環境下之辨識率的單因子變異數分析 43
表 6 個別任務動作之單因子變異數分析 44
表 7　任務平均完成時間 45
表 8 KINECT與K+S作T-TEST檢定 46
表 9 問卷信度分析 47
表 10問卷的單因子變異數分析結果 48

參考文獻

第七章參考文獻
〔1〕V. Tam and L. Ling-Shan, "Integrating the Kinect camera, gesture recognition and mobile devices for interactive discussion," Teaching, Assessment and Learning for Engineering (TALE), 2012 IEEE International Conference on, pp. H4C-11-H4C-13, Hong Kong, China, Aug. 2012.
〔2〕Ravikiran J, Kavi Mahesh, Suhas Mahishi, Dheeraj R, Sudheender S, Nitin V Pujari, "Finger Detection for Sign Language Recognition," Proceedings of the International MultiConference of Engineers and Computer Scientists, pp.489-493, Hong Kong, China, March 2009.
〔3〕C. Sun, T. Zhang, B. K. Bao, C. Xu, and T. Mei, "Discriminative Exemplar Coding for Sign Language Recognition with Kinect," Cybernetics, IEEE Transactions on, pp. 1-1, Switzerland, Lausanne, Jun 2013.
〔4〕H. Hui-Huang, T. Kang-Chun, C. Zixue, and H. Tongjun, "Posture Recognition with G-Sensors on Smart Phones," Network-Based Information Systems (NBiS), 2012 15th International Conference on, pp. 588-591, Australia ,Melbourne, Sept. 2012.
〔5〕J. E. Garrido, V. M. R. Penichet, M. D. Lozano, and L. A. Sanchez, "Mobility and memory training through movement interaction," Computer Science and Information Systems (FedCSIS), 2012 Federated Conference on, pp. 883-889, Wroclaw, Polska, Sept. 2012.
〔6〕A. Reiss and D. Stricker, "Introducing a modular activity monitoring system," Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pp. 5621-5624, Boston, MA, Aug. 2011.
〔7〕Xbox360+Kinect 官方網站〔Online〕. Available: http://www.xbox.com/zh-TW/kinect/
〔8〕R. Souvenir, A. Hajja, and S. Spurlock, "Gamesourcing to acquire labeled human pose estimation data," Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pp. 1-6, Providence, RI, June 2012.
〔9〕J. L. Raheja, A. Chaudhary, and K. Singal, "Tracking of Fingertips and Centers of Palm Using KINECT," Computational Intelligence, Modelling and Simulation (CIMSiM), 2011 Third International Conference on, pp. 248-252, Langkawi, Sept. 2011.
〔10〕L. Linwan, W. Xiaoyu, W. Linglin, and G. Tianchu, "Static Human Gesture grading based on Kinect," Image and Signal Processing (CISP), 2012 5th International Congress on, pp. 1390-1393, Chongqing, Oct. 2012.
〔11〕A. Bleiweiss, D. Eshar, G. Kutliroff, A. Lerner, Y. Oshrat, and Y. Yanai, "Enhanced interactive gaming by blending full-body tracking and gesture animation," ACM SIGGRAPH ASIA 2010 Sketches, Seoul, Republic of Korea, 2010.
〔12〕O. Patsadu, C. Nukoolkit, and B. Watanapa, "Human gesture recognition using Kinect camera," Computer Science and Software Engineering (JCSSE), 2012 International Joint Conference on, pp. 28-32, Bangkok, May 2012.
〔13〕T. Banerjee, J. Keller, M. Skubic, and E. Stone, "Day or Night Activity Recognition from Video Using Fuzzy Clustering Techniques," Fuzzy Systems, IEEE Transactions on, vol. PP, pp. 1-1, 2013.
〔14〕鍾哲民，「加速度動作辨識系統之研究及應用」，國立成功大學，碩士論文，民國97年。
〔15〕J. C. Lee, "Hacking the Nintendo Wii Remote," Pervasive Computing, IEEE, vol. 7, pp. 39-45, 2008.
〔16〕L. Johnson, S. Adams Becker, M. Cummins, V. Estrada, A. Freeman, and H. Ludgate. (2013) NMC Horizon Report > 2013 Higher Education Edition. Austin, Texas: The New Media Consortium.
〔17〕L. HistogramCheng-Hsien and L. Wei-Yang, "Human action classification using histogram-based discriminative embedding," Intelligent Signal Processing and Communications Systems (ISPACS), 2012 International Symposium on, pp. 7-11, New Taipei, Nov. 2012.
〔18〕A. W. Vieira, T. Lewiner, W. R. Schwartz, and M. Campos, "Distance matrices as invariant features for classifying MoCap data," Pattern Recognition (ICPR), 2012 21st International Conference on, pp. 2934-2937, Tsukuba, Nov. 2012.
〔19〕L. Wanqing, Z. Zhengyou, and L. Zicheng, "Expandable Data-Driven Graphical Modeling of Human Actions Based on Salient Postures," Circuits and Systems for Video Technology, IEEE Transactions on, vol. 18, pp. 1499-1510, Nov. 2008.
〔20〕C. address-eventAddress-EventShoushun, P. Akselrod, Z. Bo, J. A. Perez-Carrasco, B. Linares-Barranco, and E. Culurciello, "Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 34, pp. 302-314, Feb. 2012.
〔21〕T. Teixeira, E. Culurciello, and A. G. Andreou, "An Address-Event Image Sensor Network," Circuits and Systems, 2006. ISCAS 2006. Proceedings. 2006 IEEE International Symposium on, pp. 4467-4470, Island of Kos, May 2006.
〔22〕C. Hua-Tsung, C. Chien-Li, T. Wen-Jin, L. Suh-Yin, and Y. Jen-Yu, "Extraction and representation of human body for pitching style recognition in broadcast baseball video," Multimedia and Expo (ICME), 2011 IEEE International Conference on, pp. 1-4, Barcelona, Spain, July 2011.
〔23〕M. Kourogi, T. Ishikawa, and T. Kurata, "A method of pedestrian dead reckoning using action recognition," Position Location and Navigation Symposium (PLANS), 2010 IEEE/ION, pp. 85-89, Indian Wells, CA, USA, May 2010.
〔24〕C. Fang-Chen, J.-S. Wang, Y. Ya-Ting, and K. Tzu-Ping, "A wearable activity sensor system and its physical activity classification scheme," Neural Networks (IJCNN), The 2012 International Joint Conference on, pp. 1-6, Brisbane, QLD, June 2012.
〔25〕L. Wenfeng, B. Junrong, F. Xiuwen, G. Fortino, and S. Galzarano, "Human Postures Recognition Based on D-S Evidence Theory and Multi-sensor Data Fusion," Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on, pp. 912-917, Ottawa, May 2012.
〔26〕L. Bao and S. S. Intille, "Activity Recognition from User-Annotated Acceleration Data," Pervasive Computing, Second International Conference, pp 1-17, Linz/Vienna, Austria, April 2004.
〔27〕U. Maurer, A. Smailagic, D. P. Siewiorek, and M. Deisher, "Activity recognition and monitoring using multiple sensors on different body positions," Wearable and Implantable Body Sensor Networks, International Workshop on, pp. 4 pp.-116, Cambridge, MA , April 2006.
〔28〕J. Lester, T. Choudhury, and G. Borriello, "A practical approach to recognizing physical activities," Proceedings of the 4th international conference on Pervasive Computing, Pages 1-16, Springer-Verlag Berlin, Heidelberg, 2006.
〔29〕m. Huynh and B. Schiele, "Analyzing features for activity recognition," Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies, Pages 159 - 163, Grenoble, France, october 2005.
〔30〕K.-T. Song and C.-W. Chen, "Multi-Person Pose Recognition Using a Zigbee Sensor Network," World Congress, pp. 14976-14981, Seoul, Korea, July 2008.
〔31〕H. Harms, O. Amft, Tro, x, and G. ster, "Estimating Posture-Recognition Performance in Sensing Garments Using Geometric Wrinkle Modeling," Information Technology in Biomedicine, IEEE Transactions on, vol. 14, pp. 1436-1445, Nov. 2010.
〔32〕T.-L. Le, M.-Q. Nguyen, and T.-T.-M. Nguyen, "Human posture recognition using human skeleton provided by Kinect," Computing, Management and Telecommunications (ComManTel), 2013 International Conference on, pp. 340-345, Nanchang, Jiangxi, Jan. 2013.
〔33〕S. Monir, S. Rubya, and H. S. Ferdous, "Rotation and scale invariant posture recognition using Microsoft Kinect skeletal tracking feature," Intelligent Systems Design and Applications (ISDA), 2012 12th International Conference on, pp. 404-409, Kochi, Nov. 2012.
〔34〕F. D. Zainordin, L. Hwea Yee, N. A. Sani, W. Yong Min, and C. Chee Seng, "Human pose recognition using Kinect and rule-based system," World Automation Congress (WAC), 2012, pp. 1-6, Puerto Vallarta, Mexico, June 2012.
〔35〕X. Lu, C. Chia-Chih, and J. K. Aggarwal, "View invariant human action recognition using histograms of 3D joints," Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pp. 20-27, June 2012.
〔36〕J. Charles and M. Everingham, "Learning shape models for monocular human pose estimation from the Microsoft Xbox Kinect," Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pp. 1202-1208, Barcelona, Spain, Nov. 2011.
〔37〕J. Mazloum, A. Jalali, and J. Amiryan, "A novel bidirectional neural network for face recognition," Computer and Knowledge Engineering (ICCKE), 2012 2nd International eConference on, pp. 18-23, Mashhad, Oct. 2012.
〔38〕N. Seliya and T. M. Khoshgoftaar, "Active learning with neural networks for intrusion detection," Information Reuse and Integration (IRI), 2010 IEEE International Conference on, pp. 49-54, Las Vegas, NV, Aug. 2010.
〔39〕H. Zhiyan, L. Shuxian, and W. Jian, "A Study on Speech Emotion Recognition Based on CCBC and Neural Network," Computer Science and Electronics Engineering (ICCSEE), 2012 International Conference on, pp. 144-147, Hangzhou, March 2012.
〔40〕Robert Faludi著，and 林義翔，劉士達譯，建置無線感測網路，O’REILLY，2012年3月。
〔41〕張智傑，「帕金森症病人復健輔助系統」，國立中央大學，碩士論文，2012年6月。
〔42〕T. Kohonen, "The self-organizing map," Proceedings of the IEEE, vol. 78, pp. 1464-1480, 1990.
〔43〕蘇木春，張孝德，機器學習：類神經網路模糊系統以及基因演算法則: 全華科技圖書，2007年10月。
〔44〕曹祖聖. MSDN: Kinect for Windows 開發〔Online〕. Available: http://msdn.microsoft.com/zh-tw/hh367958.aspx
〔45〕T. DongPing and C. LianJin, "A review of the evolution of research on information Technology Acceptance Model," Business Management and Electronic Information (BMEI), 2011 International Conference on, pp. 588-591, Guangzhou , May 2011.
〔46〕Support Vector Machines 〔Online〕. Available: https://en.wikipedia.org/wiki/Support_vector_machine
〔47〕C.-C. Chang and C.-J. Lin. LIBSVM -- A Library for Support Vector Machines 〔Online〕. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm/
〔48〕k-means clustering 〔Online〕. Available: http://en.wikipedia.org/wiki/K-means_clustering

指導教授

葉士青(Shih-Ching Yeh)

審核日期

2013-8-9

推文