整合大腦與構音之類神經網路模型模擬中文字詞之產生;The neural network model integrating brain and speech model for Chinese syllables

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Electrical Engineering > Electronic Thesis & Dissertation > Item 987654321/75967

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/75967

Title:	整合大腦與構音之類神經網路模型模擬中文字詞之產生;The neural network model integrating brain and speech model for Chinese syllables
Authors:	洪國軒;Hung, Kuo-Hsuan
Contributors:	電機工程學系
Keywords:	中文聲調;Direction Into Velocities Articulator;Gradient Order DIVA;speech sound map;Chinese tone;Direction Into Velocities Articulator;Gradient Order DIVA;speech sound map
Date:	2018-01-31
Issue Date:	2018-04-13 11:24:54 (UTC+8)
Publisher:	國立中央大學
Abstract:	隨著正子放射型電腦斷層照影( PET )、功能磁共振成像( fMRI )與磁共振成像( MRI )等技術的演進，讓大腦與說話的神經關聯不再披著一層未知的面紗，即便如此，許多言語障礙依然沒有有效的治療方式。因此，本研究結合大腦與構音模型模擬中文字詞與聲調變化，而過去的構音模型多以母音為主，本模型加入特定子音模擬CV結構，最後應用在模擬語言障礙成因的假設，找出語言治療的有效方法。而本研究所使用之學習構音模型為以類神經網路為基礎的模型-DIVA ( direction into velocities articulator )，大腦訊號模型為-GODIVA ( gradient order DIVA )，DIVA模型主要模擬部位為左側運動前皮質、上顳葉皮質、下頂葉皮質、運動皮質及小腦皮質五個功能區，分別對應於語音聽覺映射( speech sound map，SSM)、聽覺狀態與誤差映射( auditory state and error map )、體感狀態與誤差映射( somatosensory state and error map )、構音器速度與位置映射( articulatory velocity and position map )以及小腦模塊( cerebellum )。而GODIVA模型則為模擬大腦左下額葉溝、額葉島蓋及前運動輔助區，分別代表語音音韻表現區、語音結構表現區以及語音聽覺映射區。因此實驗方法為找出兩模型交集區塊語音聽覺映射作為GODIVA投射至DIVA的輸入，而GODIVA部份輸出為大腦訊號指令，先將大腦指令轉變為對應的構音訊號，再利用類神經網路模型改變基頻的學習目標，並與實際聲譜圖與聲道結構做比對，在單母音聲譜圖部份，模擬結果除了/ㄨ/以外之母音都位於人聲共振峰之範圍內，但都位於範圍之邊界地帶，母音共振峰結果趨勢為往F1為450Hz、F2為1600Hz靠近。在CV結構聲道構造部份，選擇塞音與雙母音/ㄞ/結合，中文子音的塞音部分，則沒有有無聲的差別，皆為有無送氣，因此在調整上先確定舌頭位置，再調整送氣大小，模擬結果與實際構音有相同趨勢。然而受限於DIVA模型發聲構造僅分為唇、齒齦、硬顎、軟顎、小舌頭與咽六個部份，有些子音無法精確模擬，且母音的選擇也會影響子音的發聲，未來希望能將DIVA聲道模型切割得更為細部與完善，達到訊號由大腦下達指令，構音器精確模擬所有中文聲調字詞之功能。 ;With the advent of PET, fMRI and MRI, the brain function areas of speech are no longer covered with an unknown veil. Even so, there are still no effective treatments for many speech disorders. While in the past speech models were dominated by vowels, this study proposes to combine the brain and the speech model to simulate Chinese syllables and tone changes. With the integrated model, we can add designated consonants to simulate CV structure, and finally applied in the simulation of disorder hypotheses to find effective ways to treat language. In this study, the brain and speech model used were DIVA( direction into velocities articulator ) and GODIVA( gradient order DIVA ). The DIVA model contains the speech sound map (SSM), the auditory state and error map, the somatosensory state and error map, the articulatory velocity and position map, and the cerebellum, each component of the model correspond to the left anterior pre-motor cortex, the parietal and temporal cortex, the parietal lobe cortex, the motor cortex and the cerebellar cortex. The GODIVA model simulates the left inferior frontal sulcus, the frontal operculum and the pre-supplementary motor area, which respectively represent the phonological performance area, the speech structure performance area and the speech auditory mapping area. Our approach was to apply the intersection of two models, the speech auditory map, as the projection from GODIVA to DIVA. The output of the GODIVA model was used as the brain signal instruction. The first step of this study was to change the brain instruction into auditory signal, and then use the neural network model to adjust the fundamental frequency of the learning target. At last, we compared the simulation results with the actual sound spectrum and shape of the vocal tract. In the part of the vowel spectrum, the simulation results were located within the regions of typical vowel formants except for the vowel /ㄨ/ (/u/), but were all located at the boundary regions. The first formant of the tested vowels tend to approach 450 Hz and the second formants near1600 Hz. In the part of the vocal tract shape of CV structure, we select the stop consonants and diphthon /ㄞ/ (/ai/) as the CV structure. Because Chinese stop consonants have no difference in voice cue but aspiration, we only have to adjust tongue location and intensity of the aspiration. It is obvious to notice that the same trend existed between the simulation results and the actual vocal tract shapes. However, due to the fact that the speech structure of the DIVA model is divided only into six parts, including labial, alveolar ridge, hard palate, velum, uvula and pharynx, some consonants cannot be accurately simulated. Besides, the selection of the vowel affects the simulation stability. For future study, we hope that the vocal tract shape of the DIVA model could be modified to accurately simulate all Chinese tonal syllables.
Appears in Collections:	[Graduate Institute of Electrical Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	376	View/Open

社群 sharing

Loading...