以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：18

、訪客IP：3.14.144.145

姓名	賴泓誌(Hung-Chih Lai)  查詢紙本館藏	畢業系所	電機工程學系
論文名稱	應用深度學習模型偵測動作前腦波訊號 於虛擬環境中之即時人物控制 (Real-Time Character Control in Virtual Environments Using Deep Learning to Detect Pre-Movement EEG Signals)
檔案	[Endnote RIS 格式]    [Bibtex 格式]    [相關文章]   [文章引用]   [完整記錄]   [館藏目錄]   至系統瀏覽論文 (2026-6-30以後開放)
摘要(中)	因從大腦發送訊號到執行動作的過程可大致分為籌劃、準備、執行三個階段，而根據執行任務的不同所需要的執行時間也有所不同，故運動前腦波可以當作判斷動作判斷的穩定依據，本研究所偵測的腦電波包含整個執行的過程，透過分析運動前的腦波變化，訓練一個深度學習的模型，能夠根據腦波數據預測動作，克服腦波發送信號到動作執行的時間延遲，實現更高速且穩定的即時控制。 本研究採用了六通道的LSTM（長短期記憶網絡）結合Multi-head Attention機制的模型架構。LSTM負責編碼腦波序列數據，保留時間信息，而Attention機制根據序列中的重要部分進行動態加權，增強模型對運動意圖的識別能力。實驗設計包括四種運動狀態（左手、右手、雙手按鍵前進及休息），受試者在這些狀態下生成腦波數據作為訓練資料。系統分為線下與線上階段：線下階段受試者執行按鍵操作以生成運動前的腦波信號，並用於訓練模型；在線上階段會利用平滑視窗每隔0.1秒輸出一次模型的結果，並在動作的決策階段加入投票機制，以達到受試者可以利用即時腦波信號控制虛擬環境中的角色移動。 實驗結果顯示，六位參與者的線下模型平均準確率達到86.3%，並透過ERD/ERS分析驗證了運動觀察能有效激活運動相關腦波區域，增強運動準備狀態。此外，基於運動前腦波的控制系統相比傳統肌電圖系統展現了更低的延遲與更高的即時性。這項研究提供了一種創新的腦波訓練系統，未來可應用於神經康復、虛擬實境交互以及腦機介面技術，最終期望實現僅依靠腦波信號進行行為控制，而無需實際的身體運動。
摘要(英)	The process of transmitting signals from the brain to execute actions can be broadly divided into three stages: planning, preparation, and execution. The required execution time varies depending on the specific task. Thus, pre-movement brainwaves can serve as a stable basis for predicting actions. This study detects brainwaves covering the entire execution process and analyzes pre-movement brainwave changes to train a deep learning model. This model predicts actions based on brainwave data, overcoming the time delay between brain signal transmission and action execution, enabling faster and more stable real-time control. This study employs a six-channel LSTM (Long Short-Term Memory) network combined with a Multi-head Attention mechanism. The LSTM encodes brainwave sequence data, retaining temporal information, while the Attention mechanism dynamically weights critical parts of the sequence, enhancing the model′s ability to identify movement intentions. The experimental design includes four movement states (left hand, right hand, both hands pressing keys to move forward, and rest). Brainwave data generated by participants in these states served as training data. The system comprises offline and online phases: in the offline phase, participants performed key-press operations to generate pre-movement brainwave signals for model training; in the online phase, a sliding window outputs the model′s results every 0.1 seconds, and a voting mechanism is applied during the decision phase to enable participants to control a virtual environment character′s movements using real-time brainwave signals. The experimental results show that the offline models achieved an average accuracy of 86.3% across six participants. ERD/ERS analysis verified that observing movements effectively activates brain regions associated with motor activity, enhancing the state of movement preparation. Moreover, compared to traditional electromyography (EMG) systems, the brainwave-based control system demonstrated lower latency and higher real-time performance. This study provides an innovative brainwave training system with potential applications in neurorehabilitation, virtual reality interaction, and brain-computer interface technologies. Ultimately, the goal is to enable behavior control purely through brainwave signals, without requiring actual physical movement.
關鍵字(中)	★ 腦機介面 ★ 深度學習 ★ 運動前腦波	關鍵字(英)	★ Brain-Computer Interface ★ Deep Learning ★ Pre-Movement EEG
論文目次	中文摘要 i Abstract ii 目錄 iii 圖目錄 v 表目錄 vii 第一章 緒論 1 1-1 研究動機與目標 1 第二章 研究設計與方法 4 2-1 系統架構與設備 4 2-1-1 EEG量測系統與受試者量測 5 2-1-2 Unity虛擬實境系統 7 2-2 實驗流程 8 2-3 模型架構與訓練策略 11 2-3-1 長短期記憶與注意力機制 12 2-3-2 模型訓練策略 16 2-4 資料驗證 19 第三章 實驗結果與討論 21 3-1 實驗結果 21 3-1-1 線下模型訓練結果 21 3-1-2 線下模型結果比較(有無投票) 25 3-1-3 線下模型結果在不同時間點投票的比較 28 3-2 線下資料驗證 30 3-3 即時線上實驗結果 37 3-4 實驗討論 39 第四章 結論與未來展望 42 第五章 參考文獻 43
參考文獻	[1] R. Mane, T. Chouhan, and C. Guan, "BCI for stroke rehabilitation: motor and beyond," (in eng), J Neural Eng, vol. 17, no. 4, p. 041001, Aug 17 2020, doi: 10.1088/1741-2552/aba162. [2] M. Soufineyestani, D. Dowling, and A. Khan, "Electroencephalography (EEG) Technology Applications and Available Devices," Applied Sciences, vol. 10, no. 21, p. 7453, 2020. [Online]. Available: https://www.mdpi.com/2076-3417/10/21/7453. [3] A. Frisoli et al., "A New Gaze-BCI-Driven Control of an Upper Limb Exoskeleton for Rehabilitation in Real-World Tasks," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1169-1179, 2012, doi: 10.1109/TSMCC.2012.2226444. [4] V. Morash, O. Bai, S. Furlani, P. Lin, and M. Hallett, "Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries," Clin Neurophysiol, vol. 119, no. 11, pp. 2570-8, Nov 2008, doi: 10.1016/j.clinph.2008.08.013. [5] S. Bodda and S. Diwakar, "Exploring EEG spectral and temporal dynamics underlying a hand grasp movement," PLoS One, vol. 17, no. 6, p. e0270366, 2022, doi: 10.1371/journal.pone.0270366. [6] E. J. McDermott, J. Metsomaa, P. Belardinelli, M. Grosse-Wentrup, U. Ziemann, and C. Zrenner, "Predicting motor behavior: an efficient EEG signal processing pipeline to detect brain states with potential therapeutic relevance for VR-based neurorehabilitation," Virtual Real, vol. 27, no. 1, pp. 347-369, 2023, doi: 10.1007/s10055-021-00538-x. [7] H. Kuang and J. Qu, "LSTM Model with Self-Attention Mechanism for EEG Based Cross-Subject Fatigue Detection," presented at the 2021 IEEE 3rd International Conference on Frontiers Technology of Information and Computer (ICFTIC), 2021. [8] G. Zhang, V. Davoodnia, A. Sepas-Moghaddam, Y. Zhang, and A. Etemad, "Classification of Hand Movements From EEG Using a Deep Attention-Based LSTM Network," IEEE Sensors Journal, vol. 20, no. 6, pp. 3113-3122, 2020, doi: 10.1109/jsen.2019.2956998. [9] C. Spampinato, S. Palazzo, I. Kavasidis, D. Giordano, N. Souly, and M. Shah, "Deep learning human mind for automated visual classification," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6809-6817. [10] G. Pfurtscheller and F. L. Da Silva, "Event-related EEG/MEG synchronization and desynchronization: basic principles," Clinical neurophysiology, vol. 110, no. 11, pp. 1842-1857, 1999. [11] N. Laskaris, "Deep Learning and Machine Learning Decoders of EEG during Hand Movements," IKEE, 2024. [12] Q. Gao, Y. Zhang, Z. Wang, E. Dong, X. Song, and Y. Song, "Channel Projection-Based CCA Target Identification Method for an SSVEP-Based BCI System of Quadrotor Helicopter Control," Comput Intell Neurosci, vol. 2019, p. 2361282, 2019, doi: 10.1155/2019/2361282. [13] A. Jain and L. Kumar, "PreMovNet: Premovement EEG-Based Hand Kinematics Estimation for Grasp-and-Lift Task," IEEE Sensors Letters, vol. 6, no. 7, pp. 1-4, 2022, doi: 10.1109/LSENS.2022.3183284. [14] J. P. Tauscher, F. W. Schottky, S. Grogorick, P. M. Bittner, M. Mustafa, and M. Magnor, "Immersive EEG: Evaluating Electroencephalography in Virtual Reality," in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 23-27 March 2019 2019, pp. 1794-1800, doi: 10.1109/VR.2019.8797858. [15] K. Terada, A. Ikeda, T. Nagamine, and H. Shibasaki, "Movement-related cortical potentials associated with voluntary muscle relaxation," (in eng), Electroencephalogr Clin Neurophysiol, vol. 95, no. 5, pp. 335-45, Nov 1995, doi: 10.1016/0013-4694(95)00098-j. [16] A. Vourvopoulos et al., "Effects of a Brain-Computer Interface With Virtual Reality (VR) Neurofeedback: A Pilot Study in Chronic Stroke Patients," Front Hum Neurosci, vol. 13, p. 210, 2019, doi: 10.3389/fnhum.2019.00210. [17] D. Wu, H. Wan, S. Liu, W. Yu, Z. Jin, and D. Wang, "DeepBrain: Towards personalized EEG interaction through attentional and embedded LSTM learning," arXiv preprint arXiv:2002.02086, 2020. [18] Y. Zhao and D. Chen, "Expression EEG Multimodal Emotion Recognition Method Based on the Bidirectional LSTM and Attention Mechanism," Comput Math Methods Med, vol. 2021, p. 9967592, 2021, doi: 10.1155/2021/9967592. [19] C. C. LinFeng, Mingyan Zhao, Huiyuan Deng, and Yong Zhang, "EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism," 2022. [20] D. L. Eaves, M. Riach, P. S. Holmes, and D. J. Wright, "Motor Imagery during Action Observation: A Brief Review of Evidence, Theory and Future Research Opportunities," (in eng), Front Neurosci, vol. 10, p. 514, 2016, doi: 10.3389/fnins.2016.00514. [21] T. Najafi, R. Jaafar, R. Remli, and W. A. Wan Zaidi, "A Classification Model of EEG Signals Based on RNN-LSTM for Diagnosing Focal and Generalized Epilepsy," (in eng), Sensors (Basel), vol. 22, no. 19, Sep 25 2022, doi: 10.3390/s22197269. [22] P. Nagabushanam, S. Thomas George, and S. Radha, "RETRACTED ARTICLE: EEG signal classification using LSTM and improved neural network algorithms," Soft Computing, vol. 24, no. 13, pp. 9981-10003, 2020/07/01 2020, doi: 10.1007/s00500-019-04515-0. [23] X. Hu, S. Yuan, F. Xu, Y. Leng, K. Yuan, and Q. Yuan, "Scalp EEG classification using deep Bi-LSTM network for seizure detection," Computers in Biology and Medicine, vol. 124, p. 103919, 2020. [24] N. Michielli, U. R. Acharya, and F. Molinari, "Cascaded LSTM recurrent neural network for automated sleep stage classification using single-channel EEG signals," Computers in biology and medicine, vol. 106, pp. 71-81, 2019. [25] X. Zheng and W. Chen, "An attention-based bi-LSTM method for visual object classification via EEG," Biomedical Signal Processing and Control, vol. 63, p. 102174, 2021. [26] A. Craik, Y. He, and J. L. Contreras-Vidal, "Deep learning for electroencephalogram (EEG) classification tasks: a review," Journal of neural engineering, vol. 16, no. 3, p. 031001, 2019. [27] G. Pfurtscheller and F. H. Lopes da Silva, "Event-related EEG/MEG synchronization and desynchronization: basic principles," (in eng), Clin Neurophysiol, vol. 110, no. 11, pp. 1842-57, Nov 1999, doi: 10.1016/s1388-2457(99)00141-8. [28] K. Lakshminarayanan, R. Shah, S. R. Daulat, V. Moodley, Y. Yao, and D. Madathil, "The effect of combining action observation in virtual reality with kinesthetic motor imagery on cortical activity," (in English), Frontiers in Neuroscience, Original Research vol. 17, 2023-June-13 2023, doi: 10.3389/fnins.2023.1201865. [29] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, "Brain-computer interfaces for communication and control," (in eng), Clin Neurophysiol, vol. 113, no. 6, pp. 767-91, Jun 2002, doi: 10.1016/s1388-2457(02)00057-3. [30] M. Pratiwi, A. D. Wibawa, and M. H. Purnomo, "EEG-based happy and sad emotions classification using LSTM and bidirectional LSTM," in 2021 3rd international conference on electronics representation and algorithm (ICERA), 2021: IEEE, pp. 89-94. [31] K. Martin-Chinea, J. Ortega, J. Gomez-Gonzalez, E. Pereda, J. Toledo, and L. Acosta, "Effect of time windows in LSTM networks for EEG-based BCIs," Cognitive Neurodynamics, vol. 17, no. 2, pp. 385-398, 2023. [32] M. M. Abdulghani, W. L. Walters, and K. H. Abed, "Imagined Speech Classification Using EEG and Deep Learning," (in eng), Bioengineering (Basel), vol. 10, no. 6, May 26 2023, doi: 10.3390/bioengineering10060649.
指導教授	李柏磊 徐國鎧(Po-Lei Lee Kuo-Kai Shyu)	審核日期	2025-1-22
推文	facebook   plurk   twitter   funp   google   live   udn   HD   myshare   reddit   netvibes   friend   youpush   delicious   baidu
網路書籤	Google bookmarks   del.icio.us   hemidemi   myshare