以視覺為基礎的人機互動系統中，手部通常為離攝影機最近的物體，因此藉由3D資訊可以有效偵測手部區塊。我們利用粒子最佳化(PSO)演算法，加強連續影像的手部區塊偵測的穩定性與強健性。本研究提出一個雙模態的動態手勢辨識方法。第一個模態(MIP)為運動歷史影像(Motion History Image, MHI)、影像矩(Image Moment)特徵擷取與機率神經網路(Probability Neural Network, PNN)的手勢辨識方法；第二個模態(FIS)根據軌跡變化的手勢模糊推論系統(Fuzzy Inference System)進行手勢辨識，最後透過決策融合，融合雙模態各手勢推論機率，得到最佳辨識效果。實驗結果顯示，透過立體視覺與PSO追蹤，能夠有效且精準找出手部區塊；而經由決策融合後的雙模態動態手勢辨識方法其辨識結果皆優於單一模態辨識方法的辨識結果。此一連續三維手勢辨識提供了新世代人機互動系統一個人性化、友善的互動技術。In vision-based human-computer interaction system, hand region is usually the nearest object to cameras, therefore, it is effective to detect hand region by 3D information. We use Particle Swarm Optimization (PSO) algorithm to enhance the stability and robustness of continuous image hand region detection. This paper proposes a dual modal dynamic gesture recognition method. The first one (MIP) consists of motion history image (MHI), image moment feature extraction and probability neural network (PNN), the second one (FIS) is the fuzzy inference system based on trajectory variation. At last, through decision fusion, we fuse the probabilities of gestures from dual modal methods to get the best recognition result. From the experiments, the hand region is detected efficiently and precisely by stereo vision and PSO tracking, the recognition rate of dual modal method is also better than two single modal methods. This continuous 3D gesture recognition provides a reliable, friendly interactive technology in the new generation of HCI system.
