ChatGPT-Driven Fake Review Detection: Assessing the Impact of Review Aspects

NCUIR > School of Management at National Central University > Graduate Institute of Information Management > Electronic Thesis & Dissertation > Item 987654321/95581

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/95581

Title:	ChatGPT-Driven Fake Review Detection: Assessing the Impact of Review Aspects
Authors:	林介元;LIN, CHIEH-YUAN
Contributors:	資訊管理學系
Keywords:	假評論偵測;文字探勘;監督式學習;遷移式學習;生成式 AI;Fake review detection;Text mining;Supervised learning;Transfer learning;Generative AI
Date:	2024-07-29
Issue Date:	2024-10-09 17:04:28 (UTC+8)
Publisher:	國立中央大學
Abstract:	線上顧客評論在電子商務中扮演著至關重要的角色，假評論的出現對電子商務生態系統造成了負面影響，因此，開發有效的假評論偵測方法成為了一個重要的研究課題。近年來，雖然有一些研究使用生成式 AI 來生成假評論並分析其生成機制，然而，這些研究大多僅著重於潤飾或改寫現有評論，並未對假評論的多種面向進行深入探討。因此，本研究旨在開發有效的假評論偵測方法，並分析由 ChatGPT 生成的各種假評論面向對偵測表現的影響。本研究使用 YelpZip 資料集，通過文本預處理、多種特徵提取方法、重新採樣技術，並結合多種機器學習與深度學習進行實驗，如隨機森林、eXtreme Gradient Boosting (XGBoost)、邏輯迴歸、Bidirectional Encoder Representations from Transformers (BERT)及 Robustly Optimized BERT Approach (RoBERTa)，以找出最有效的假評論偵測方法。本研究使用 YelpZip 資料集與 ChatGPT 生成不同種類的假評論，包含單純潤飾後的假評論、含有不同餐廳資訊的假評論以及含有不同用餐經驗的假評論，深入分析不同面向的評論對假評論偵測效果的影響。研究結果顯示，在傳統機器學習方法中，使用 BERT 嵌入、隨機過採樣和邏輯迴歸組合達到最佳表現 (AUC：0.715)。在深度學習方法中，RoBERTa 表現最佳 (AUC：0.770)。根據不同假評論面向的偵測結果，說明 ChatGPT 修飾後的假評論較容易辨識以及評論中餐廳資訊與用餐經驗的取得程度對於假評論偵測的影響。本研究運用 ChatGPT 潤飾與生成不同面向的假評論，深入分析其對假評論偵測效能的影響，為假評論偵測研究提供新的視角與思路。;Online customer reviews play a crucial role in e-commerce, but the emergence of fake reviews negatively impacts the e-commerce ecosystem. Developing effective methods for fake reviews detection (FRD) has become an important research domain. Recent studies using generative AI to create fake reviews focus on embellishing existing reviews. This research aims to devise efficacious FRD methods and evaluate the effects of different aspects of fake reviews generated by ChatGPT on detection performance. We use the YelpZip dataset and apply multiple feature extraction methods, resampling techniques, and combining various machine learning (ML) and deep learning (DL) approaches, such as Random Forest, eXtreme Gradient Boosting (XGBoost), Logistic Regression (LR), Bidirectional Encoder Representations from Transformers (BERT), and Robustly Optimized BERT Approach (RoBERTa), to find the most effective method. Different types of fake reviews were generated using ChatGPT, including simply embellished fake reviews, those containing different restaurant information, and reviews with various dining experiences. We deeply analyze how various review aspects impact FRD performance. The results show that among ML methods, the combination of BERT embeddings, Random Oversampling (ROS), and LR achieved the best performance (AUC: 0.715). In DL methods, RoBERTa performed the best (AUC: 0.770). The detection results suggest that ChatGPT-rephrased fake reviews are easier to identify, and the inclusion of restaurant information and dining experiences in the reviews impacts the performance of FRD. This study leverages ChatGPT to rephrase and create fake reviews with varying aspects, providing new perspectives and valuable insights in fake reviews detection.
Appears in Collections:	[Graduate Institute of Information Management] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	80	View/Open

社群 sharing

Loading...