基於立體視覺的連續三維手勢辨識

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：44

、訪客IP：3.136.19.124

姓名

徐瑩珊(Ying-shan Hsu) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於立體視覺的連續三維手勢辨識
(Continuous 3D Gesture Recognition Based on Stereo Vision)

相關論文

★ 整合GRAFCET虛擬機器的智慧型控制器開發平台	★ 分散式工業電子看板網路系統設計與實作
★ 設計與實作一個基於雙攝影機視覺系統的雙點觸控螢幕	★ 智慧型機器人的嵌入式計算平台
★ 一個即時移動物偵測與追蹤的嵌入式系統	★ 一個固態硬碟的多處理器架構與分散式控制演算法
★ 基於立體視覺手勢辨識的人機互動系統	★ 整合仿生智慧行為控制的機器人系統晶片設計
★ 嵌入式無線影像感測網路的設計與實作	★ 以雙核心處理器為基礎之車牌辨識系統
★ 微型、超低功耗無線感測網路控制器設計與硬體實作	★ 串流影像之即時人臉偵測、追蹤與辨識─嵌入式系統設計
★ 一個快速立體視覺系統的嵌入式硬體設計	★ 即時連續影像接合系統設計與實作
★ 基於雙核心平台的嵌入式步態辨識系統	★ Gigabit乙太網路的UDP/IP硬體加速器設計

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

以視覺為基礎的人機互動系統中，手部通常為離攝影機最近的物體，因此藉由3D資訊可以有效偵測手部區塊。我們利用粒子最佳化(PSO)演算法，加強連續影像的手部區塊偵測的穩定性與強健性。本研究提出一個雙模態的動態手勢辨識方法。第一個模態(MIP)為運動歷史影像(Motion History Image, MHI)、影像矩(Image Moment)特徵擷取與機率神經網路(Probability Neural Network, PNN)的手勢辨識方法；第二個模態(FIS)根據軌跡變化的手勢模糊推論系統(Fuzzy Inference System)進行手勢辨識，最後透過決策融合，融合雙模態各手勢推論機率，得到最佳辨識效果。實驗結果顯示，透過立體視覺與PSO追蹤，能夠有效且精準找出手部區塊；而經由決策融合後的雙模態動態手勢辨識方法其辨識結果皆優於單一模態辨識方法的辨識結果。此一連續三維手勢辨識提供了新世代人機互動系統一個人性化、友善的互動技術。

摘要(英)

In vision-based human-computer interaction system, hand region is usually the nearest object to cameras, therefore, it is effective to detect hand region by 3D information. We use Particle Swarm Optimization (PSO) algorithm to enhance the stability and robustness of continuous image hand region detection. This paper proposes a dual modal dynamic gesture recognition method. The first one (MIP) consists of motion history image (MHI), image moment feature extraction and probability neural network (PNN), the second one (FIS) is the fuzzy inference system based on trajectory variation. At last, through decision fusion, we fuse the probabilities of gestures from dual modal methods to get the best recognition result. From the experiments, the hand region is detected efficiently and precisely by stereo vision and PSO tracking, the recognition rate of dual modal method is also better than two single modal methods. This continuous 3D gesture recognition provides a reliable, friendly interactive technology in the new generation of HCI system.

關鍵字(中)

★ 立體視覺
★ 手勢辨識
★ PSO
★ MHI
★ 人機互動

關鍵字(英)

★ stereo vision
★ gesture recognition
★ PSO
★ MHI
★ HCI

論文目次

摘要 i
Abstract ii
誌謝 iii
目錄 iv
圖目錄 vi
表目錄 viii
第一章緒論 1
1.1 研究動機 1
1.2 研究內容與系統架構 2
1.3 論文架構 3
第二章相關文獻探討 5
2.1 人機互動 5
2.2 立體視覺 7
2.3 物體追蹤 8
2.4 手勢識別 12
第三章基於立體視覺的手掌區域定位 16
3.1 影像前處理 16
3.1.1 膚色偵測 17
3.1.2 型態學 18
3.1.3 連通元件 22
3.2 雙攝影機之立體視覺 24
3.3 結合粒子最佳化追蹤手部區塊 30
3.3.1 PSO-based追蹤 30
3.3.2 搜尋空間與搜尋方框 33
3.3.3 目標物特徵與適應函數 34
3.3.4 動態建模 38
第四章動態手勢辨識 41
4.1 MIP推論動態手勢類別 41
4.1.1 運動歷史影像(Motion-History Images, MHI) 42
4.1.2 MHI特徵向量擷取 44
4.1.3 機率神經網路(Probabilistic Neural Network, PNN)分類器 46
4.2動態手勢的模糊推論系統(Fuzzy Inference System, FIS) 52
4.2.1 手勢軌跡特徵擷取 52
4.2.2 模糊神經網路分類器 54
4.2.3 基於FIS的動態手勢細分類 61
4.3 決策融合 63
第五章系統設計與實驗 65
5.1 軟硬體開發環境 65
5.1.1 硬體設備 65
5.1.2 操作與軟體開發環境 66
5.2 手部區塊追蹤 68
5.2.1 距離攝影機最近的膚色物體偵測 68
5.2.2 超出攝影機交集區外 70
5.2.3 與其他膚色物體距離攝影機距離相近 70
5.2.4 手部追蹤實驗比較 72
5.3 動態手勢辨識 73
5.3.1 單一手勢 74
5.3.2 混合手勢 77
5.4 訓練參數 79
5.4.1 MIP機率神經網路的σ 79
5.4.2 FIS模糊神經網路分類器權重值 79
5.4.3 決策融合權重 80
第六章結論與未來方向 82
6.1 結論 82
6.2 未來方向 83
參考文獻 84

參考文獻

[1] [Online.] WIKIPEDIA website, “Human-computer interaction”,“http://en.wikipedia.org/wiki/Human-computer_interaction”
[2] [Online.] BBC CHINESE.com, Jan 2008, “蓋茨：未來五年人機互動將發生變革”, http://news.bbc.co.uk/chinese/trad/hi/newsid_7170000/newsid_7174800/7174801.stm.
[3] C.X. Nian, 2009, Design and Implementation of a Dual-touch Display Based on Dual-camera Vision System, Master Thesis, National Central University, Department of Computer Science and Information Engineering, page 3-12.
[4] A. Królak, and P. Strumiłło, 2008, “Vision-Based Eye Blink Monitoring System for Human-Computer Interfacing”, IEEE conference on Human System Interactions, pp.994-998.
[5] W. H. Wang, 2009, “An Intelligent User-Friendly Human-Computer Interface for Home Appliance”, Master Thesis, National Central University, Department of Computer Science and Information Engineering, page 10-13.
[6] C.Y Huang, 2008, “Implement a real-time Human Face/Sound Source Tracking and Speech Purification System”, Master Thesis, National Chiao Tung University, Institute of Electrical Control Engineering, page 3-24.
[7] L.W. Ko, 2007, “Neural Human Machine Interface and Its Applications”, Ph.D Thesis, National Chiao Tung University, Institute of Electrical Control Engineering,
page 1-13.
[8] K. H. Chang, 2007, “A study on the economic polarized light stereoscopic projection system”, Master Thesis, National Central University, Department of Optics and Photonics, page 7-14.
[9] Y. P. Chang, 2003, “A Real-Time Object Tracking System Using the Stereo Vision”, Master Thesis, Chung Yuan Christian University, Department of Mechanical Engineering, page 4.
[10] D. Koller, K .Daniilidis, and H.Nagel, 1993, "Model-based object tracking in monocular image sequences of road traffic scenes", International Journal of Computer Vision, pp. 257-281.
[11] I. Karaulova, P. Hall, and A. Marshall , 2000, "A hierarchical model of dynamics for tracking people with a single video camera", Proc. of British Machine Vision Conference, pp. 262-352.
[12] [Online.]W.H.Yao, M.X.Tsai, I.X. Chen and S.C.Wang, Mar 2005, “Introduction of Object Tracking System”, “http://vbie.eic.nctu.edu.tw/vol_2/skill_6.htm”.
[13] A. Blake and M. Isard, 1994, “3D Position, Altitude and Shape Input Using Video Tracking of Hands and Lips,” Proc. Computer Graphics, SIGGRAPH, pp. 71-78.
[14] S. Birchfield and S. Rangarajan, 2005, “Spatiograms versus histograms for region-based tracking”, Proc. of IEEE Conference on Computer Vision and Pattern Recognition , Volume 2, pp. 1158-1163, 20-25.
[15] K. Nummiaro, K. M. Esther and L. V. Gool, 2003, "An Adaptive Color-Based Particle Filter," Proc. of Image And Vision Computing, Volume 21, pp. 99-110.
[16] C. R. Wren, A. Azarbayejani, T. Darrell, and A. P. Pentland, 1997, “Pfinder: real-time tracking of the human body,” IEEE Trans. Pattern Anal. Machine Intell., Volume 19, pp. 780-785.
[17] S. T. Birchfield and S. Rangarajan, 2007, “Spatial Histograms for Region-Based Tracking”, ETRI Journal, Volume 29, No 5, pp. 697-699.
[18] I. A. Karaulova, P. M. Hall, and A. D. Marshall, 2000, “A hierarchical model of dynamics for tracking people with a single video camera,” Proc.British Machine Vision Conf., pp. 262-352.
[19] M. Kass, A. WitKin and D. Terzopoulos, 1987, “Snake: Active contour models,”International Journal of Computer Vision, Volume 1, pp.321-331.
[20] M. K. Chien, 2007, “Enhanced Active Contour Models for Tracking Shape-Changing Hand Gestures in Taiwanese Sign Language”, Master Thesis, National Cheng Kung University, Department of Computer Science and Information Engineering, page 12-16.
[21] J. L. Lai, 2006, “A Study on Visual Detection and Tracking of Moving Targets”, Master Thesis, National Cheng Kung University, Department of Electrical Engineering, page 24-39.
[22] R. Bourezak and G. Bilodeau, 2006, “Object detection and tracking using iterative division and correlograms”, Proc. of the 3rd Canadian Conference of Computer and Robot Vision, pp. 38.
[23] Y. Hayashi, H. Fujiyoshi, 2008, “Mean-shift-based color tracking in illuminance change”, Artificial Intelligence Lecture, Volume 5001, pp.302-311.
[24] G. Welch and G. Bishop, 2006, “An introduction to the kalman filter,” University of North Carolina at Chapel Hill, Department of Computer Science, Tech. Rep. 95-041.
[25] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, 2002, “A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking”, IEEE Trans. Signal Processing, Volume 50, No. 2, pp. 174-188.
[26] T. Zhang, S. Fei, X.D. Li and H. Lu, 2008, “An Improved Particle Filter for Tracking Color Object”, Proc. of ICICTA, Volume 02, pp.109-113.
[27] Y. Satoh, T. Okatani and K. Deguchi, 2004, “A Color-based Tracking by Kalman Particle Filter”, Proc. of the 17th International Conference on Pattern Recognition, Volume 3, pp. 502-505, 23-26.
[28] M. Isard and A. Blake , 1998, “CONDENSATION – Conditional Density Propagation for Visual Tracking”, International Journal on Computer Vision 1, Volume 29, pp. 5-28.
[29] C. Manresa, J. Varona, R. Mas and F. Perales, 2005, “Hand tracking and gesture recognition for human-computer interaction”, ELCVIA, Volume 5, No. 3, p.96-104
[30] W. T. Freeman and M. Roth, 1995, “ Orientation Histograms for Hand Gesture Recognition”, IEEE Intl. Wkshp. on Automatic Face and Gesture Recognition, Volume 50, Issue 2, pp.174.
[31] K. J. Chang, 2005, “Computer Vision Based Hand Gesture Recognition System”,Master Thesis, National Tsing Hua University, Department of Electrical Engineering, page 21-32 .
[32] L.K. Chang, 2002, “Hand Gesture Recognition Based on Hausdorff Distance”, Journal of Image and Graphics, Volume 7, No.11.
[33] M. Elmezain, A. Al-Hamadi, B. Michaelis, 2009, “Hand Trajectory-based Gesture Spotting and Recognition Using HMM”, ICIP 16th IEEE International Conference , pp. 3577-3580.
[34] A. Corradini, 2001, “Dynamic TimeWarping for Off-line Recognition of a Small Gesture Vocabulary”, Proc. of IEEE ICCV Wkshp, pp.82-89.
[35] K. Murakami and H. Taguchi, 1991, “Gesture Recognition using Recurrent Neural Networks”, Proc. of the SIGCHI conference on Human factors in computing systems, pp.237-242.
[36] W. C. Lin, 2009, “Skin Color Detection and Face Location in Different sceneries”, Master Thesis, National Central University, Department of Computer Science and Information Engineering, page 8-17.
[37] R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, 2001, “Face detection in color images,” International Conference on Image Processing, pp. 1046-1049.
[38] T. M. Mahmoud, 2008 , “A New Fast Skin Color Detection Technique”, World Academy of Science, pp.501-505.
[39] S.Z. Huang, 2009, “Human-Machine Interaction Using Stereo Vision-based Gesture Recognition”, Master Thesis, National Central University, Department of Computer Science and Information Engineering, page 10-23.
[40] J. Kennedy and R. Eberhart, 1995, "Particle swarm optimization", Proc. of IEEE International Conference on Neural Networks, Volume 4, pp.1942-1948.
[41] B.M. Mehtre , M. S. Kankanhalli , A. D. Narasimhalu , and G. C. Man, 1995, “Color matching for image retrieval”, Pattern Recognition Letters, pp. 325-331.
[42] M.C. Yen, 2009, “An Embedded System for Real-Time Detection and Tracking of Moving Object”, Master Thesis, National Central University, Department of Computer Science and Information Engineering, page 26-31.
[43] D. M. Gavrila and S. Munder, 2007, “Multi-cue pedestrian detection and tracking from a moving vehicle,” International Journal of Computer Vision, Volume. 73,
No.1, pp. 41-59.
[44] [Online.]鄭琇文, 2009, “科技始於人性徹底顛覆人機介面 Canesta讓手勢即可操控電視夢想得以實現”, “http://www.digitimes.com.tw/tw/dt/n/shwnws.asp?CnlID=13&Cat=&Cat1=&id=12680 0”.
[45] A. Bobick, J. Davis, 2001, “The recognition of human movement using temporal templates”, IEEE Trans. Pattern Anal. Mach. Intell., Volume 23, No.3, pp.257-267.
[46] C. Shan, T. Tan,Y. Wei, 2007, “Real-time hand tracking using a mean shift embedded particle filter”, Pattern Recognition, Volume 40, Issue 7, pp.1958-1970.
[47] Y.A. Pan, 2004, “Automatic Facial Expression Recognition System in Low Resolution Image Sequence”, Master Thesis, National Cheng Kung University, Department of Computer Science and Information Engineering, page 33-34.
[48] R.C. Gonzalez, R.E. Woods, 繆紹綱編譯, 2006, 數位影像處理, 培生教育出版集團
[49] R.C. Gonzalez, R.E. Woods, 1992, Digital Image Processing, Addison-Wesley.
[50] Zaknich, 1998, “Introduction to the Modified Probabilistic Neural Network for General Signal Processing Applications”, IEEE Trans. On Signal Processing, Volume 46. No. 7, pp. 1980-1990.
[51] D. F. Specht, 1990, “Probabilistic Neural Networks (original contribution)”, Neural Networks, Volume 3, No.1, pp.109-118.
[52] F. J. Lin, W. J. Hwang, and R. J. Wai, 1999, “A supervisory fuzzy neural network control system for tracking periodic inputs,” IEEE Trans. Fuzzy Systems, Volume 7, No.1, pp. 41-52.
[53] Y. C. Chen and C. C. Teng, 1995, “A model reference control structure using a fuzzy neural network,” Fuzzy Sets and Systems, Volume 73, pp.291-312.
[54] [Online.]林進燈, “類神經網路”,
http://www.ecaa.ntu.edu.tw/weifang/LifeScience/FuzzyNeuro.html
[55] J.H Lu, 2005, “Design and Hardware Implementationof a Secure Bi-Modal Biometric System”, Master Thesis, I-Shou University , Department of Electrical Engineering, page 32-43.
[56] [Online.]D. Takahashi, “CES: Does the future of TV lie in gesture-based control?"
http://games.venturebeat.com/2009/01/11/ces-does-the-future-of-tv-lie-in-gesture-based- control/
[57] 蘇木春, 張孝德, 2006, 機器學習：類神經網路、模糊系統以及基因演算法則, 全華科技圖書股份有限公司, 陳本源.
[58] M.K. Hu, 1962, “Visual pattern recognition by moment invariants”, IEEE Trans. Information Theory, Volume 8, pp.179-187

指導教授

陳慶瀚(Ching-han Chen)

審核日期

2010-7-10

推文