多模態特徵融合的維度心理素質分析虛擬心理師;Virtual Psychologist for Dimensional Psychological Quality Analysis with Multi-Modal Feature Fusion

NCUIR > College of Electrical Engineering & Computer Science > Software Engineer > Electronic Thesis & Dissertation > Item 987654321/92285

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/92285

Title:	多模態特徵融合的維度心理素質分析虛擬心理師;Virtual Psychologist for Dimensional Psychological Quality Analysis with Multi-Modal Feature Fusion
Authors:	龔彥安;Yen-An, Kung,
Contributors:	軟體工程研究所
Keywords:	虛擬人;自動性格識別;多模態融合;Virtual Human;Automatic Personality Recognition;Multi-Modal Fusion
Date:	2023-07-24
Issue Date:	2023-10-04 15:25:25 (UTC+8)
Publisher:	國立中央大學
Abstract:	心理素質是心理健康的內生因素。心理素質的主要方法是基於使用問捲和量表，這些方法成本高且耗時。近期的研究表示，文本、語音、面部表情特徵、心率與眼睛運動可適用於心理素質測評。在本文中，我們建立了一套用於移動設備自動心理質量評估的虛擬治療師，可以藉由語音對話主動進行引導用戶且使用情緒偵測方法改變談話內容。在談話過程中，從文本、語音、面部表情、心率和眼睛運動中提取特徵，用於多模態心理質量評估。我們使用兩個融合框架進行自動心理質量分析和機器學習，對不同的維度和因素集進行分類，包括抑鬱、身體、興奮、不穩定、焦慮、家庭護理、獨立、憂鬱、躁狂和焦慮，以及家庭關係。結合 168 名受試者的結果數據，實驗結果表示，使用五種模態特徵的融合框架的總準確率達到了抑鬱、身體、興奮、不穩定、焦慮、家庭照顧、獨立、憂鬱傾向、躁狂傾向、焦慮傾向的最高準確率, 和家庭關係分別達到 68.66%、74.66%、72.06%、93.65%、70.66%、72.66%、93.33%、68.66%、84.43%、70.66%、72%。 ;Psychological quality plays a crucial role in mental health and is typically assessed through the use of questionnaires and scales, which can be expensive and time-consuming. However, recent research has shown promising alternatives for assessing psychological quality. These include analyzing various sources such as text, audio, facial attributes, heart rate, and eye movement. In this paper, we propose the development of a virtual therapist specifically designed for automatic psychological quality assessment on mobile devices. This virtual therapist would actively engage users in voice dialogue, adapting the conversation content based on emotion perception. Throughout the conversation, we extract features from multiple modalities including text, audio, facial attributes, heart rate, and eye movement, enabling a comprehensive assessment of psychological quality. We utilize two fusion frameworks for automatic psychological quality analysis and machine learning to classify the varying sets of dimensions and factors, which include depression, body, excitement, instability, anxiety, family care, independence, melancholic, manic, and anxiety as well as the family relationship. Based on the data collected from 168 participants, the experimental results demonstrate the effectiveness of our fusion framework utilizing five modal features. The highest accuracy rates were achieved for various psychological factors including depression, body, excitement, instability, anxiety, family care, independence, melancholic tendencies, manic tendencies, anxiety tendencies, and family relationship. Specifically, the fusion framework achieved accuracy rates of 68.66 percent, 74.66 percent, 72.06 percent, 93.65 percent, 70.66 percent, iii 72.66 percent, 93.33 percent, 68.66 percent, 84.43 percent, 70.66 percent, and 72 percent, respectively, for these factors. These findings highlight the robustness and reliability of our approach in accurately assessing and predicting various aspects of psychological well-being.
Appears in Collections:	[Software Engineer] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	114	View/Open

社群 sharing

Loading...