以多特徵神經網路實現連續手語識別;Realizing Sign Language Recognition using Multi-Feature Neural Network

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/89795

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/89795

Title:	以多特徵神經網路實現連續手語識別;Realizing Sign Language Recognition using Multi-Feature Neural Network
Authors:	費群安;Fitriajie, Arda Satata
Contributors:	資訊工程學系
Keywords:	圖像處理;視頻處理;連續手語識別;手勢識別;關鍵點;Image Processing;Video Processing;Continuous Sign Language Recognition;Gesture Recognition;Keypoint
Date:	2022-07-25
Issue Date:	2022-10-04 12:00:02 (UTC+8)
Publisher:	國立中央大學
Abstract:	若有 RGB 視頻串流，我們的目標是正確識別與連續手語識別 (CSLR) 相關的手語。儘管該領域提出的深度學習方法逐漸增加，但大多數主要集中在僅使用 RGB 特徵，無論是全幀圖像還是手部和臉部的細節。 CSLR 訓練過程信息的不足嚴重限制了他們學習視頻輸入幀中多個特徵的能力。目前，多特徵網路變得相當普遍，因為當前的計算能力不再限制我們擴大網路規模。因此，在本文中，我們將研究深度學習網路並應用多特徵技術，以期增加和改進當前的連續手語識別任務，詳細說明我們將包括的另一個特徵在這項研究中，如果我們將它們做比較，關鍵點特徵沒有圖像特徵那麼沉重。這項研究的結果表明，在 Phoenix2014 和中國手語這兩個最流行的 CSLR 數據集上，添加關鍵點特徵作為一種多特徵模態可以提高識別率，或者通常會降低單詞錯誤率 (WER)。 ;Given the RGB video streams, we aim to recognize signs related to continuous sign language recognition (CSLR) correctly. Despite there are increasing of proposed deep learning methods in this area, most of them mainly focus on only using an RGB feature, either the full frame image or the detail of hands and face. The scarcity of information for the CSLR training process heavily constrains their capability to learn the multiple features within the video input frames. Currently, Multi-feature networks became something quite common since the current computing power is something that is not limiting us from scaling the network size anymore. Thus, in this thesis, we’re going to work deep learning network and apply a multi-feature technique with the hope to increase & improve the current state of the art of continuous sign language recognition tasks, in detail another feature that we would include in this research is the key-point feature which is not as heavy as the image feature if we are comparing them. The result of this research shows that adding a key-point feature as a multi-feature modality could increase the recognition rate or commonly, decrease the word error rate (WER) on the two most popular CSLR datasets: Phoernix2014 and Chinese Sign Language.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	71	View/Open

社群 sharing

Loading...