使用大型語言模型構建自動化問答系統;Automated Question-Answering System Using Large Language Models

NCUIR > College of Electrical Engineering & Computer Science > Software Engineer > Electronic Thesis & Dissertation > Item 987654321/98036

Please use this identifier to cite or link to this item: https://ir.lib.ncu.edu.tw/handle/987654321/98036

Title:	使用大型語言模型構建自動化問答系統;Automated Question-Answering System Using Large Language Models
Authors:	呂宗祐;Lyu, Zong-You
Contributors:	軟體工程研究所
Keywords:	大型語言模型;自動資料集生成;開源模型;提示工程;問答系統;思維鏈;Large Language Model;Automatic Dataset Generation;Open- Source Model;Prompt Engineering;Question-Answering System;Chain of Thought
Date:	2025-08-04
Issue Date:	2025-10-17 12:16:45 (UTC+8)
Publisher:	國立中央大學
Abstract:	語言模型資料集的人工標註一直是一件耗費人力的事，但隨著最近幾年開源大語言模型不斷的更迭，有越來越多人使用大型語言模型來協助資料集的產生。因此，本研究提出了一個全開源模型的架構，並以Gemma 2-27B模型來做為主要的語言模型，目的是能夠自動化的產生語言模型的訓練資料集，以達到節省人力的目的，並提升語言模型在量化標準上的表現。本研究將會驗證在進行微調和進行RAG的排列組合中，何種訓練方法的量化分數會最高，並會在實驗過程中加入思考鏈會不會提升量化分數。並且本研究將會以餘弦相似度和LLM-as-judge的指標來做為評量，並會與市面上的其他資料集作為比較。最後，會將此系統藉由ngrok技術部屬於Line bot上，以實現人機互動介面以及利用Prompt實現的簡易MCP Tool Calling，並能夠通過UI來靈活切換模型。;Manual generation of datasets for language models has long been a labor-intensive task. However, with the rapid evolution of open-source large language models in recent years, more and more researchers have begun leveraging LLMs to assist in dataset generation. Therefore, this study proposes a fully open-source architecture that leverages the Gemma 2-27B model as the core language model. The primary goal is to automate the generation of training datasets for large language models, thereby reducing human effort and improving performance on quantitative evaluation metrics. This research will explore which training strategies across combinations of fine-tuning and retrieval-augmented generation (RAG) will yield the highest quantitative scores. It will also examine whether incorporating chain-of-thought (CoT) reasoning during generation improves the results. Evaluation will be conducted using cosine similarity and LLM-as-a-judge metrics, and results will be compared against existing public datasets. Finally, the system will be deployed to a LINE Bot via ngrok, enabling a human-AI interactive interface and a lightweight MCP tool calling mechanism using prompt-based control. The user interface will also support dynamic model switching for flexible operation.
Appears in Collections:	[Software Engineer] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	46	View/Open

社群 sharing

Loading...