隨著取像設備的價格大幅的降低，監控系統目前已經廣泛的應用在日常生活當中。不過目前使用的監控系統大多都只有錄影的功能，只能提供事後的資訊。因此便有人提出了智慧型監控系統的概念，利用電腦視覺的方法，在不需要人為的操作之下，讓監控系統能夠自動對攝影機所擷取的影像進行分析，以具有偵測、追蹤、辨識、分析的功能。 本論文提出一個限制區域非法進入者偵測系統，利用制服色彩為特徵，用以判斷進入限制區域中的人員是否具有合法之身分。首先利用背景相減法來偵測是否有目標物的存在，並利用目標物的位置、大小與色彩等資訊來追蹤物體。接著使用一個以區塊為主的身體區域分割演算法，將目標物切割為頭部、上半身與下半身區域。最後針對穿著制服的身體區域抽取色彩的特徵抽並分類，以判斷該進入者是否具有合法之身分。 實驗結果顯示，本論文所提出之方法可以有效的辨別進入者之身分。 Due to the cost-down of capturing devices, surveillance systems are gradually widely applied in our daily life. However, the main function of current surveillance systems only focus on the recoding of video data. Besides, a lot of attention has to be paid by the surveillants in monitoring the video data. The developing of an automatic and intelligent surveillance system to detect, track, recognize, and analyze moving objects is an effective solution for saving the human resources. The main purpose of this thesis is to detect illegal entrants in restricted areas. Since a legal entrant in restricted areas always wears uniform, the color information of uniform is extracted to serve as the feature for determining whether an entrant is legal or not. Firstly, background subtraction technique is employed to detect moving objects from image sequences. Three key features including object position, object size, and object color are extracted to track the detected object. After that, the body of entrant is segmented into three regions; head, upper body and lower body, using the watershed segmentation methods. Finally, color features extracted from the region of interesting (ROI) are utilized to classify the legality of an entrant. Experiments were conducted to verify the feasibility and validity of our proposed system in detecting and tracking illegal entrants in restricted areas. The results is satisfactory.