應用二維多頻帶特徵與深度學習網路於腦電波認知負荷評估

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：76

、訪客IP：3.137.210.110

姓名

白士弘(Shih-Hung Pai) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

應用二維多頻帶特徵與深度學習網路於腦電波認知負荷評估
(Evaluation of cognitive load level using 2D multi-frequency band EEG feature with deep learning neural network)

相關論文

★ 使用梳狀濾波器於相位編碼之穩態視覺誘發電位腦波人機介面	★ 應用電激發光元件於穩態視覺誘發電位之腦波人機介面判斷
★ 智慧型手機之即時生理顯示裝置研製	★ 多頻相位編碼之閃光視覺誘發電位驅動大腦人機介面
★ 以經驗模態分解法分析穩態視覺誘發電位之大腦人機界面	★ 利用經驗模態分解法萃取聽覺誘發腦磁波訊號
★ 明暗閃爍視覺誘發電位於遙控器之應用	★ 使用整體經驗模態分解法進行穩態視覺誘發電位腦波遙控車即時控制
★ 使用模糊理論於穩態視覺誘發之腦波人機介面判斷	★ 利用正向模型設計空間濾波器應用於視覺誘發電位之大腦人機介面之雜訊消除
★ 智慧型心電圖遠端監控系統	★ 使用隱馬可夫模型於穩態視覺誘發之腦波人機介面判斷與其腦波控制遙控車應用
★ 使用類神經網路於肢體肌電訊號進行人體關節角度預測	★ 使用等階集合法與影像不均勻度修正於手指靜脈血管影像切割
★ 應用小波編碼於多通道生理訊號傳輸	★ 結合高斯混合模型與最大期望值方法於相位編碼視覺腦波人機介面之目標偵測

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 ( 永不開放)

摘要(中)

人腦被認為是具有不同精神狀態（例如休息狀態，活動狀態或認知狀態）的複雜系統。眾所皆知，大腦活動隨著認知需求的增加而增加。而觀察認知狀態的常見方式之一是腦電圖（EEG）訊號。了解認知負荷的程度在教育研究中對於教學效益具有重要意義。過去，認知負荷程度的分級，一般都是利用精心設計好的刺激實驗，例如：N-back test。在本文中，我們提出了客觀的認知負荷量測技術，應用物理試題解題狀態於認知負荷分析。首先，我們使用方位等距投影（AEP）技術將腦波帽電極的三維（3D）坐標投影到二維（2D）平面中，並內插功率譜密度（PSD）值，將EEG時間序列轉換為承載空間訊息的二維圖像。然後，我們使用卷積神經網絡（CNN）從中提取特徵，這些特徵被傳遞到長短期記憶（LSTM）以提取EEG訊號的時間特性。這個分析流程的好處是它保留了頻譜，空間和時間結構，並提取了對各個維度變化不太敏感的特徵。實驗的結果表明，N-back test在兩個不同級別上進行認知負荷預測，準確率達到80.38％。最後，利用N-back test預訓練好的模型進行遷移學習，並對物理試題進行預測，結果發現，由於物理試題複雜性太高，模型在預測主觀難易度的準確率僅能達到55.56％，但在觀察時程圖中發現腦負荷狀態在最初及最後階段有符合我們所預期的結果。

摘要(英)

The human brain is a complex system with different mental states (such as a resting state, active state, or cognitive state). It is widely known that the brain activity increases with the increased cost of cognitive demands. To monitor the level of cognitive demand in humman brain, an effective method is the use of Electroencephalography (EEG). Especially, understanding the level of cognitive load is significantly important to measure the effects of educational measures in eduation environments. In the past studies, the detection of cognitive load level was studied under well-designed simulation experiments, such as the N-back test. However, the designs of these stimulation experiments are very much different from classroom circumstances which have difficulty in reflecting the true cognitive load levels of students in real learning environments. In this paper, we propose an objective technique for cognitive load measurements and the effectiness of the proposed method has been applied to detect the levels of cognitive load in solving the physical test questions. Firstly, we use the azimuthal isometric projection (AEP) technique to project the three-dimensional (3D) coordinates of the EEG cap electrode onto a two-dimensional (2D) plane with the values of power spectral density (PSD), in order to convert the EEG time sequence into a two-dimensional brain topographic image. The convolutional neural networks (CNN) was then applied to extract features from the data and these features will be transmitted to the long short-term memory (LSTM) for extracting the time characteristics of the EEG signals. The advantage of this analysis process is that it preserves the spectrum, space and time structure, as well as extracts features that are less sensitive to the variation in different dimensions. In our study resutls, we have successfully applied the CNN-LSTM neural netwok architecture for evaluating the cognitive load levels in N-back test with 80.38% accuracy, in the detection of two cognitive load levels (easy and hard). Finally, the N-back test pre-trained model was used for transfer learning and the physics test questions were predicted. It was found that due to the high complexity of the physics test questions, the accuracy of the model in predicting subjective difficulty was only 55.56%, but In the observation time chart, it is found that the brain load state has the results that meet our expectations in the initial and final stages.

關鍵字(中)

★ 腦電圖
★ 認知負荷
★ 物理試題
★ 卷積神經網路
★ 長短期記憶
★ 遷移學習

關鍵字(英)

★ EEG
★ Cognitive load
★ Physical test questions
★ Convolutional neural network(CNN)
★ Long short-term memory(LSTM)
★ Transfer learning

論文目次

中文摘要 i
Abstract ii
目錄 iv
圖目錄 vi
表目錄 viii
第一章緒論 1
1-1 研究背景 1
1-2 研究動機 2
1-3 文獻回顧 3
1-4 論文架構 4
第二章原理介紹與系統架構 5
2-1 原理介紹 5
2-1-1 腦波分類 5
2-1-2 大腦分區 7
2-1-3 認知負荷理論 8
2-1-4 方位等距投影（Azimuthal Equidistant Projection） 9
2-1-5 Clough-Tocher技術 10
2-2 系統架構與硬體設備 11
2-2-1 乾式電極與腦波帽 12
2-2-2 八通道腦波機 13
2-2-3 10-20電極配置法（10-20 system） 14
第三章研究設計與網路訓練 15
3-1 研究設計 15
3-1-1 受試者 15
3-1-2 通道挑選 16
3-1-3 N-back 17
3-1-4 物理試題 20
3-2 前處理 23
3-2-1 巴特沃斯濾波器（Butterworth filter） 23
3-2-2 快速傅立葉轉換（FFT） 24
3-2-3 產生EEG多通道時間序列的圖片 25
3-3 神經網路模型 27
3-3-1 卷積神經網路（CNN） 28
3-3-2 長短期記憶（LSTM） 29
3-4 網路訓練 32
3-5 遷移學習 33
第四章結果與討論 35
4-1 前處理比較 35
4-1-1 投影插值圖與時頻圖比較結果 35
4-1-2 2D與3D圖片比較結果 37
4-2 模型比較結果 39
4-3 遷移學習結果 42
第五章結論與未來展望 46
第六章參考文獻 47
附錄物理試題資料 52

參考文獻

[1] James P. Byrnes and Nathan A. Fox, “The educational relevance of research in cognitive neuroscience,” Educational Psychology Review, Vol. 10, No. 3, pp. 297-342, 1998.
[2] Piotr Mirowski, Deepak Madhavan, Yann LeCun, and Ruben Kuzniecky, “Classification of patterns of EEG synchronization for seizure prediction,” Clinical neurophysiology, Vol. 120, No. 11, pp. 1927-1940, 2009.
[3] Nihal Fatma Güler, Elif Derya Übeyli, and Inan Güler, “Recurrent neural networks employing Lyapunov exponents for EEG signals classification,” Expert systems with applications Vol. 29, No.3, pp. 506-514, 2005.
[4] Abdul Qayyum, M.K.A.Ahamed Khan, Moona Mazher, and M Suresh,“Classification of EEG Learning and Resting States using 1D-Convolutional Neural Network for Cognitive Load Assesment,” IEEE Student Conference on Research and Development (SCOReD), Selangor, Malaysia, 2018.
[5] Samuel Goldstein, Zhenhong Hu, and Mingzhou Ding, “Decoding Working Memory Load from EEG with LSTM Networks,” arXiv:1910.05621, 2019.
[6] Shiba Kuanar, Vassilis Athitsos, Nityananda Pradhan, Arabinda Mishra, and K.R.Rao, “Cognitive analysis of working memory load from EEG, by a deep recurrent neural network,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, 2018.
[7] Anushri Saha, Vikash Minz, Sanjith Bonela, S. R. Sreeja, Ritwika Chowdhury, and Debasis Samanta, “Classification of eeg signals for cognitive load estimation using deep learning architectures,” International Conference on Intelligent Human Computer Interaction, pp. 59-68, Springer, Cham, Nov. 2018.
[8] Michal Teplan,“Fundamentals of EEG measurement,” Measurement science review, Vol. 2, No. 2, pp. 1-11, 2002.
[9] John Sweller, “Cognitive load during problem solving: Effects on learning,” Cognitive science, Vol. 12, No. 2, pp. 257-285, 1988.
[10] John Parr Snyder, Map projections--A working manual, Vol. 1395. US Government Printing Office, 1987.
[11] Peter Alfeld, “A trivariate clough—tocher scheme for tetrahedral data,” Computer Aided Geometric Design, Vol. 1, No. 2, pp. 169-181, 1984.
[12] Tian Lan, Deniz Erdogmus, Andre Adami, Santosh Mathan, and Misha Pavel , “Channel selection and feature projection for cognitive load estimation using ambulatory EEG,” Hindawi Publishing Corporation Computational Intelligence and Neuroscience Volume 2007, Article ID 74895, 12 pages doi:10.1155/2007/74895.
[13] Janet P. Trammell, Priscilla G. MacRae, Greta Davis, Dylan Bergstedt and Ariana E. Anderson, “The relationship of cognitive performance and the theta-alpha power ratio is age-dependent: An eeg study of short term memory and reasoning during task and resting-state in healthy young and old adults,” Frontiers in aging neuroscience, Vol. 9, pp. 364, 2017.
[14] Kirchner, Wayne K., “Age differences in short-term retention of rapidly changing information,” Journal of experimental psychology, Vol. 55, No. 4, pp. 352, 1958.
[15] Ron Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” International Joint Conference on Articial Intelligence (IJCAI), 1995.

[16] Prajoy Podder, Md. Mehedi Hasan, Md. Rafiqul Islam, and Mursalin Sayeed, “Design and implementation of Butterworth, Chebyshev-I and elliptic filter for speech signal analysis,” arXiv:2002.03130, 2020.
[17] James W. Cooley and John W. Tukey, “An algorithm for the machine calculation of complex Fourier series,” Mathematics of computation, Vol. 19, No. 90, pp. 297-301, 1965.
[18] Amjed S. Al-Fahoum and Ausilah A. Al-Fraihat, “Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains,” Hindawi Publishing Corporation ISRN Neuroscience Volume 2014, Article ID 730218, 7 pages http://dx.doi.org/10.1155/2014/730218.
[19] Pouya Bashivan, Irina Rish, Mohammed Yeasin, and Noel Codella, “Learning representations from EEG with deep recurrent-convolutional neural networks,” arXiv:1511.06448, 2015.
[20] Mehmet Akif Ozdemir , Murside Degirmenci , Onan Guren, and Aydin Akan, “EEG based Emotional State Estimation using 2-D Deep Learning Technique,” Medical Technologies Congress (TIPTEKNO), Izmir, Turkey, Oct. 2019.
[21] Pouya Bashivan, Irina Rish, Mohammed Yeasin, and Noel Codella., “Learning representations from EEG with deep recurrentconvolutional neural networks,” arXiv:1511.06448, 2015.
[22] Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga, and George Toderici, “Beyond short snippets: Deep networks for video classification,” Proceedings of the IEEE conference on computer vision and pattern recognition(CVPR), pp. 4694-4702, 2015.
[23] Karen Simonyan and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv:1409.1556, 2014.
[24] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams, “Learning representations by back-propagating errors,” Nature, Vol. 323, pp. 533-536, 1986.
[25] Sepp Hochreiter and Jürgen Schmidhuber, “Long short-term memory,” Neural computation, Vol. 9, No. 8, pp. 1735-1780, 1997.
[26] Martin Wöllmer, Florian Eyben, Alex Graves, Björn Schuller and Gerhard Rigoll, “Bidirectional LSTM networks for context-sensitive keyword detection in a cognitive virtual agent framework,” Cognitive Computation, Vol. 2, pp. 180-190, 2010.
[27] Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed, “Hybrid speech recognition with deep bidirectional LSTM,” IEEE workshop on automatic speech recognition and understanding, Olomouc, Czech Republic, Dec. 2013.
[28] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems(NIPS), 2012.
[29] Tejas Dharamsi, Payel Das, Tejaswini Pedapati, Gregory Bramble, Vinod Muthusamy, Horst Samulowitz, Kush R. Varshney, Yuvaraj Rajamanickam, John Thomas, and Justin Dauwels, “Neurology-as-a-Service for the Developing World,” arXiv:1711.06195, 2017.
[30] Vedang Mandhana and Rutuja Taware, “Classification of 3D interpolated EEG signals using hybrid R-3DCNN,” Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 17-19, Brno Czech Republic, Mar. 2020.

指導教授

李柏磊(Po-Lei Lee)

審核日期

2020-10-14

推文