| dc.description.abstract | According to statistics from the Taiwan′s Ministry of Health and Welfare in 2021,
There are about 1,198,000 people with physical disability certificates in Taiwan,
including 125,764 people with hearing impairment.
About 5% of the total population.
The hearing-impaired people often have many difficulties in oral pronunciation and learning due to the hearing impairment since childhood.
Therefore, sign language is often used as its main communication method.
And many sign language users are reading TV news, election debates and live press conferences, etc.
Media that relies heavily on hearing to absorb information can often only be read in subtitles, including election debates,
Regular meetings of public programs organized by the government such as epidemic prevention press conferences
Equipped with a sign language teacher, the sign language teacher will convert the content of the speaker′s spoken language into the form of sign language,
Make it easier for sign language users to understand the content.
However, because the number of sign language teachers is still limited, they can only be deployed in a few occasions. then,
How to enable the hearing-impaired to have the same experience as ordinary listeners,
A major issue for modern media.
This research combines technologies from two major fields in deep learning, natural language processing and gesture recognition.
Developed a system that can perform sign language translation in time and use virtual characters to make sign language gestures,
Using the 3D gesture recognition model to convert the single-word video of sign language into a gesture data set,
Use a third-party speech recognition service to recognize the user′s spoken language and convert it into Chinese sentences,
And use the natural language processing model to convert Chinese sentences into sign language word sequences,
And compared the sign language word sequence with the gesture data set,
Then, the correct sign language gestures are passed to the avatar, so that they can make sign language gestures,
Then connect all the stages into a complete user system.
A system that enables instant interpretation of sign language.
In addition, this study also experimented and applied a variety of signal smoothing techniques,
Improve the Temporal Jitter problem common in gesture recognition,
The virtual characters can be closer to real people when they perform sign language gestures. | en_US |