應用二維多頻帶特徵與深度學習網路於腦電波認知負荷評估;Evaluation of cognitive load level using 2D multi-frequency band EEG feature with deep learning neural network

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Electrical Engineering > Electronic Thesis & Dissertation > Item 987654321/85065

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/85065

Title:	應用二維多頻帶特徵與深度學習網路於腦電波認知負荷評估;Evaluation of cognitive load level using 2D multi-frequency band EEG feature with deep learning neural network
Authors:	白士弘;Pai, Shih-Hung
Contributors:	電機工程學系
Keywords:	腦電圖;認知負荷;物理試題;卷積神經網路;長短期記憶;遷移學習;EEG;Cognitive load;Physical test questions;Convolutional neural network(CNN);Long short-term memory(LSTM);Transfer learning
Date:	2020-10-14
Issue Date:	2021-03-18 17:33:08 (UTC+8)
Publisher:	國立中央大學
Abstract:	人腦被認為是具有不同精神狀態（例如休息狀態，活動狀態或認知狀態）的複雜系統。眾所皆知，大腦活動隨著認知需求的增加而增加。而觀察認知狀態的常見方式之一是腦電圖（EEG）訊號。了解認知負荷的程度在教育研究中對於教學效益具有重要意義。過去，認知負荷程度的分級，一般都是利用精心設計好的刺激實驗，例如：N-back test。在本文中，我們提出了客觀的認知負荷量測技術，應用物理試題解題狀態於認知負荷分析。首先，我們使用方位等距投影（AEP）技術將腦波帽電極的三維（3D）坐標投影到二維（2D）平面中，並內插功率譜密度（PSD）值，將EEG時間序列轉換為承載空間訊息的二維圖像。然後，我們使用卷積神經網絡（CNN）從中提取特徵，這些特徵被傳遞到長短期記憶（LSTM）以提取EEG訊號的時間特性。這個分析流程的好處是它保留了頻譜，空間和時間結構，並提取了對各個維度變化不太敏感的特徵。實驗的結果表明，N-back test在兩個不同級別上進行認知負荷預測，準確率達到80.38％。最後，利用N-back test預訓練好的模型進行遷移學習，並對物理試題進行預測，結果發現，由於物理試題複雜性太高，模型在預測主觀難易度的準確率僅能達到55.56％，但在觀察時程圖中發現腦負荷狀態在最初及最後階段有符合我們所預期的結果。;The human brain is a complex system with different mental states (such as a resting state, active state, or cognitive state). It is widely known that the brain activity increases with the increased cost of cognitive demands. To monitor the level of cognitive demand in humman brain, an effective method is the use of Electroencephalography (EEG). Especially, understanding the level of cognitive load is significantly important to measure the effects of educational measures in eduation environments. In the past studies, the detection of cognitive load level was studied under well-designed simulation experiments, such as the N-back test. However, the designs of these stimulation experiments are very much different from classroom circumstances which have difficulty in reflecting the true cognitive load levels of students in real learning environments. In this paper, we propose an objective technique for cognitive load measurements and the effectiness of the proposed method has been applied to detect the levels of cognitive load in solving the physical test questions. Firstly, we use the azimuthal isometric projection (AEP) technique to project the three-dimensional (3D) coordinates of the EEG cap electrode onto a two-dimensional (2D) plane with the values of power spectral density (PSD), in order to convert the EEG time sequence into a two-dimensional brain topographic image. The convolutional neural networks (CNN) was then applied to extract features from the data and these features will be transmitted to the long short-term memory (LSTM) for extracting the time characteristics of the EEG signals. The advantage of this analysis process is that it preserves the spectrum, space and time structure, as well as extracts features that are less sensitive to the variation in different dimensions. In our study resutls, we have successfully applied the CNN-LSTM neural netwok architecture for evaluating the cognitive load levels in N-back test with 80.38% accuracy, in the detection of two cognitive load levels (easy and hard). Finally, the N-back test pre-trained model was used for transfer learning and the physics test questions were predicted. It was found that due to the high complexity of the physics test questions, the accuracy of the model in predicting subjective difficulty was only 55.56%, but In the observation time chart, it is found that the brain load state has the results that meet our expectations in the initial and final stages.
Appears in Collections:	[Graduate Institute of Electrical Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	106	View/Open

社群 sharing

Loading...