表情在日常生活中扮演著很重要的角色，是一種非語言的溝通方式，因此表情辨識成為許多專家學者在研究發展的議題。本論文主要是發展一套自動化的表情辨識系統，可透過擷取數位攝影機的影像，自動化地偵測人臉、擷取特徵到表情辨識。藉由人臉偵測、雙眼偵測、特徵區域的概念、特徵點的選取與光流追蹤的方法，再加入有限狀態機的機制，可以有效率且快速地組成一套自動化的表情辨識系統。 本系統採用偵測雙眼來精確的定位人臉特徵區域，提出改良式的自我組織特徵映射圖演算法，能自動化且即時地做臉部特徵點的追蹤與選取。並採用兩階段鄰域機制及相關係數光流追蹤法，提供快速地人臉特徵點的追蹤方式，藉由這些特徵點在臉部的移動判別人臉的表情。根據以上的方法，建構出本論文的表情辨識系統，在各個表情資料庫都呈現相當好的成效。最後，再結合有限狀態機的機制，將連續的序列影像自動化分割出許多表情序列影像段，讓系統可以透過數位攝影機即時判別使用者的表情動作，以達到自動化即時表情辨識系統。 Visual communication is very important in daily lives for humans as social beings. Especially, facial expressions can reveal lots of information without the use of a word. Automatic facial expression recognition systems can be applied to many practical applications such as human-computer interaction, stress-monitoring systems, low-bandwidth videoconferencing, human behavior analysis, etc. Thus in recent years, the research of developing automatic facial expression recognition systems has attracted a lot of attention from varied fields. The goal of this thesis is to develop an automatic facial expression recognition system which can automatically detect human faces, extract features, and recognize facial expressions. The inputs to the proposed automatic facial expression recognition algorithm are a sequence of images since dynamic images can provide more information about facial expressions than a single static image. After a human face is detected, the system first detects eyes and then accurately locates the regions of face features. The movements of facial points (eyebrows, eyes, and mouth) have a strong relation to the information about the shown facial expression; however, the extraction of facial features sometimes is a very challenging task. A modified self-organizing feature map algorithm is developed to automatically and effectively extract feature points. Then we adopt a two-stage neighborhood-correlation optical flow tracking algorithm to track the human feature points. The optical flow information of the feature points is used for the facial expression recognition. Most importantly, a segmentation method based on a finite state machine is proposed to automatically segment a video stream into units of facial expressions. Each segmented unit is then input to the recognition module to decide which facial expression exists in the corresponding unit. Experiments were conducted to test the performance of the proposed facial recognition system.
