運用鏡像運動資料之遷移學習於想像運動為基礎的即時回饋腦波人機介面

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：12

、訪客IP：3.15.190.144

姓名

張子謙(Tzu-Chien Chang) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

運用鏡像運動資料之遷移學習於想像運動為基礎的即時回饋腦波人機介面
(Transfer Learning of real-time Motor Imagery BCI using Action Observation EEG data with Transformer-based Spatial-Temporal Fearure)

相關論文

★ 使用梳狀濾波器於相位編碼之穩態視覺誘發電位腦波人機介面	★ 應用電激發光元件於穩態視覺誘發電位之腦波人機介面判斷
★ 智慧型手機之即時生理顯示裝置研製	★ 多頻相位編碼之閃光視覺誘發電位驅動大腦人機介面
★ 以經驗模態分解法分析穩態視覺誘發電位之大腦人機界面	★ 利用經驗模態分解法萃取聽覺誘發腦磁波訊號
★ 明暗閃爍視覺誘發電位於遙控器之應用	★ 使用整體經驗模態分解法進行穩態視覺誘發電位腦波遙控車即時控制
★ 使用模糊理論於穩態視覺誘發之腦波人機介面判斷	★ 利用正向模型設計空間濾波器應用於視覺誘發電位之大腦人機介面之雜訊消除
★ 智慧型心電圖遠端監控系統	★ 使用隱馬可夫模型於穩態視覺誘發之腦波人機介面判斷與其腦波控制遙控車應用
★ 使用類神經網路於肢體肌電訊號進行人體關節角度預測	★ 使用等階集合法與影像不均勻度修正於手指靜脈血管影像切割
★ 應用小波編碼於多通道生理訊號傳輸	★ 結合高斯混合模型與最大期望值方法於相位編碼視覺腦波人機介面之目標偵測

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-8-1以後開放)

摘要(中)

腦機介面(Brain Computer Interface, BCI)提供神經或是肌肉損傷的人士一個利用腦電波與外界交流的管道。過去要求受試者進行想像運動，然而對於癱瘓病人很難對動作進行想像，因此我們以鏡像神經元系統(human Mirror Neuron System, hMNS)作為訓練想像運動的方法，結合VR(virtual reality)觀看影片建立想像依據，分別建立個人的模型(individual model)、共同的模型(global model)，接著讓受試者回想VR場景進行想像運動，最後建立一個回溯想像的即時回饋腦機介面，使想像運動的訓練過程更方便、容易。
實驗分成三個部份，第一部份是建立鏡像神經觀察腦波模型，第二部份建立回溯想像遷移學習模型，第三部份利用前面訓練出的模型建立BCI即時回饋系統。實驗中將乾式腦波電極設置在 10-20 EEG System 之 F3、F4、 C3、Cz、C4、P3、Pz、P4的位置，模型需要兩秒的腦波資料，並以每0.1秒的移動視窗(sliding window)，利用動作觀察、回溯想像、即時回饋加強後的模型，進行即時BCI的判斷。
實驗受試者有五人，皆為年齡介於20~22的健康男性，慣用手皆為右手。結果顯示在個人的模型上，觀察模型的平均準確率達到55.8%，回溯想像的平均準確率達到63.6%，即時回饋的平均準確率達到73.1%。在共同的模型上，觀察模型的平均準確率達到50%，回溯想像的平均準確率達到61%，即時回饋的平均準確率達到66.5%。

摘要(英)

Brain Computer Interface (BCI) provides a way for people with nerve or muscle damage to communicate with the outside world using brain waves. In the past, subjects were required to perform motor imagery. However, it is difficult for paralyzed patients to do motor imagery. Therefore, we used the human Mirror Neuron System (hMNS) as a method to train motor imagery, combined with VR (virtual reality) to watch videos establish an imaginary basis, respectively establish an individual model and a global model, and then ask the subjects to recall the VR scene to perform motor imagery, and finally establish a real-time feedback brain-computer interface for retrospective imagination, so that the motor imagery training process is simpler and more convenient.
The experiment is divided into three parts, the first part is to establish a mirror nerve action observation brain wave model, the second part is to establish a retrospective imaginative transfer learning model, and the third part uses the previously trained model to establish a BCI real-time feedback system. In the experiment, the dry brainwave electrodes are set at the positions of F3, F4, C3, Cz, C4, P3, Pz, and P4 of the 10-20 EEG System. The model requires two seconds of brainwave data, and the sliding window moves every 0.1 seconds, using action observation, retrospective imagination, and real-time feedback to strengthen the model to make real-time BCI judgments.
There were five subjects in the experiment, all of them were healthy males between the ages of 20 and 22, and all of them were right-handed. The results show that on the idividual model, the average accuracy of the observation model is 55.8%, the average accuracy of retrospective imagination is 63.6%, and the average accuracy of instant feedback is 73.1%. On the global model, the average accuracy of the observation model is 50%, the average accuracy of retrospective imagination is 61%, and the average accuracy of instant feedback is 66.5%.

關鍵字(中)

★ 腦電波
★ 腦機介面
★ 想像運動
★ 鏡像神經元
★ 深度神經網路

關鍵字(英)

論文目次

中文摘要 v
Abstract vi
目錄 viii
圖目錄 x
表目錄 xii
第一章緒論 1
1-1研究動機與目的 1
1-2 文獻探討 2
1-3 論文章節架構 4
第二章原理介紹 5
2-1 腦電訊號 5
2-1-1 常用腦電訊號成分 5
2-1-2 大腦皮質區域及功能 6
2-1-3 腦電波測量方法與位置 7
2-2 腦電波分析方法 8
2-2-1 事件相關去同步(ERD)、事件相關同步(ERS) 8
2-2-2 共空間模式(Common Spatial Pattern，CSP) 9
2-3 腦機介面 11
2-4 Transformer 為基礎的腦波分類網路 12
第三章研究設計與方法 15
3-1 系統架構 15
3-1-1 實驗設備 16
3-2 實驗流程 17
3-2-1觀察腦波模型建立流程 17
3-2-2回溯想像腦波模型建立流程 18
3-2-3即時回饋腦波人機介面實驗流程 19
3-2-4受試者資料 20
第四章結果與討論 21
4-1 動作觀察、想像運動ERD/ERS特徵 21
4-2 觀察腦波模型訓練結果 27
4-3 回溯想像腦波模型建立 28
4-4 即時反饋實驗 30
4-5 資料分析(ERD/ERS) 40
4-6 分類器比較結果 53
第五章結論與未來展望 66
第六章參考文獻 68

參考文獻

[1] Graimann, B., Allison, B. Z., & Pfurtscheller, G. (Eds.). (2010). Brain-computer interfaces: Revolutionizing human-computer interaction. Springer Science & Business Media.
[2] Lance, B. J., Kerick, S. E., Ries, A. J., Oie, K. S., & McDowell, K. (2012). Brain–computer interface technologies in the coming decades. Proceedings of the IEEE, 100(Special Centennial Issue), 1585-1599.
[3] Guan, C., Thulasidas, M., & Wu, J. (2004, December). High performance P300 speller for brain-computer interface. In IEEE International Workshop on Biomedical Circuits and Systems, 2004. (pp. S3-5). IEEE.
[4] Schwartz, A. B., Cui, X. T., Weber, D. J., & Moran, D. W. (2006). Brain-controlled interfaces: movement restoration with neural prosthetics. Neuron, 52(1), 205-220.
[5] Käthner, I., Wriessnegger, S. C., Müller-Putz, G. R., Kübler, A., & Halder, S. (2014). Effects of mental workload and fatigue on the P300, alpha and theta band power during operation of an ERP (P300) brain–computer interface. Biological psychology, 102, 118-129.
[6] Abdulkader, S. N., Atia, A., & Mostafa, M. S. M. (2015). Brain computer interfacing: Applications and challenges. Egyptian Informatics Journal, 16(2), 213-230.
[7] Shih, J. J., Krusienski, D. J., & Wolpaw, J. R. (2012, March). Brain-computer interfaces in medicine. In Mayo clinic proceedings (Vol. 87, No. 3, pp. 268-279). Elsevier.
[8] Tanji, K., Sakurada, K., Funiu, H., Matsuda, K., Kayama, T., Ito, S., & Suzuki, K. (2015). Functional significance of the electrocorticographic auditory responses in the premotor cortex. Frontiers in Neuroscience, 9, 78.
[9] Guger, C., Edlinger, G., Harkam, W., Niedermayer, I., & Pfurtscheller, G. (2003). How many people are able to operate an EEG-based brain-computer interface (BCI)?. IEEE transactions on neural systems and rehabilitation engineering, 11(2), 145-147.
[10] Blankertz, B., Sannelli, C., Halder, S., Hammer, E. M., Kübler, A., Müller, K. R., ... & Dickhaus, T. (2010). Neurophysiological predictor of SMR-based BCI performance. Neuroimage, 51(4), 1303-1309.
[11] Jiang, A., Shang, J., Liu, X., Tang, Y., Kwan, H. K., & Zhu, Y. (2020). Efficient CSP algorithm with spatio-temporal filtering for motor imagery classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(4), 1006-1016.
[12] Bhattacharyya, S., Khasnobish, A., Chatterjee, S., Konar, A., & Tibarewala, D. N. (2010, December). Performance analysis of LDA, QDA and KNN algorithms in left-right limb movement classification from EEG data. In 2010 International conference on systems in medicine and biology (pp. 126-131). IEEE.
[13] Wu, W., Gao, X., & Gao, S. (2006, January). One-versus-the-rest (OVR) algorithm: An extension of common spatial patterns (CSP) algorithm to multi-class case. In 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference (pp. 2387-2390). IEEE.
[14] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444.
[15] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
[16] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
[17] Nijboer, F., Furdea, A., Gunst, I., Mellinger, J., McFarland, D. J., Birbaumer, N., & Kübler, A. (2008). An auditory brain–computer interface (BCI). Journal of neuroscience methods, 167(1), 43-50.
[18] Sollfrank, T., Ramsay, A., Perdikis, S., Williamson, J., Murray-Smith, R., Leeb, R., ... & Kübler, A. (2016). The effect of multimodal and enriched feedback on SMR-BCI performance. Clinical Neurophysiology, 127(1), 490-498.
[19] Ishihara, W., Moxon, K., Ehrman, S., Yarborough, M., Panontin, T. L., & Nathan-Roberts, D. (2020, December). Feedback modalities in brain–computer interfaces: a systematic review. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 64, No. 1, pp. 1186-1190). Sage CA: Los Angeles, CA: SAGE Publications.
[20] Ziadeh, H., Gulyas, D., Nielsen, L. D., Lehmann, S., Nielsen, T. B., Kjeldsen, T. K. K., ... & Knoche, H. (2021). “Mine Works Better”: Examining the Influence of Embodiment in Virtual Reality on the Sense of Agency During a Binary Motor Imagery Task With a Brain-Computer Interface. Frontiers in Psychology, 12.
[21] Alimardani, M., Nishio, S., & Ishiguro, H. (2016). The importance of visual feedback design in BCIs; from embodiment to motor imagery learning. PloS one, 11(9), e0161945.
[22] i Badia, S. B., Morgade, A. G., Samaha, H., & Verschure, P. F. M. J. (2012). Using a hybrid brain computer interface and virtual reality system to monitor and promote cortical reorganization through motor activity and motor imagery training. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(2), 174-181.
[23] Achanccaray, D., Pacheco, K., Carranza, E., & Hayashibe, M. (2018, October). Immersive virtual reality feedback in a brain computer interface for upper limb rehabilitation. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 1006-1010). IEEE.

[24] Škola, F., & Liarokapis, F. (2018). Embodied VR environment facilitates motor imagery brain–computer interface training. Computers & Graphics, 75, 59-71.
[25] Jo, S., & Choi, J. W. (2018, January). Effective motor imagery training with visual feedback for non-invasive brain computer interface. In 2018 6th International Conference on Brain-Computer Interface (BCI) (pp. 1-4). IEEE.
[26] Neuper, C., Scherer, R., Wriessnegger, S., & Pfurtscheller, G. (2009). Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain–computer interface. Clinical neurophysiology, 120(2), 239-247.
[27] Yang, S., Lopez, S., Golmohammadi, M., Obeid, I., & Picone, J. (2016, December). Semi-automated annotation of signal events in clinical EEG data. In 2016 IEEE Signal Processing in Medicine and Biology Symposium (SPMB) (pp. 1-5). IEEE.
[28] Li, Y., Guan, C., Li, H., & Chin, Z. (2008). A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller system. Pattern Recognition Letters, 29(9), 1285-1294.
[29] Pfurtscheller, G., & Da Silva, F. L. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical neurophysiology, 110(11), 1842-1857.
[30] Rojas, G. M., Alvarez, C., Montoya, C. E., De la Iglesia-Vaya, M., Cisternas, J. E., & Gálvez, M. (2018). Study of resting-state functional connectivity networks using EEG electrodes position as seed. Frontiers in neuroscience, 12, 235.
[31] Blausen Medical. (2014). Medical gallery of Blausen medical 2014. WikiJournal of Medicine, 1(2), 1-79.
[32] Song, Y., Jia, X., Yang, L., & Xie, L. (2021). Transformer-based spatial-temporal feature learning for eeg decoding. arXiv preprint arXiv:2106.11170.

指導教授

李柏磊

審核日期

2022-9-22

推文