| 摘要: | 隨著全球學術產出規模不斷擴張，傳統的同行審查制度正面臨嚴重的人力瓶頸與審查品質不一等挑戰。由於審查作業本質上是一項耗時且高度仰賴專業判斷的工作，學界對於自動化輔助審查系統的需求日益殷切。然而，現有語言模型雖具初步生成能力，但在準確性、可解釋性與偏見控制方面仍與人類專家存在顯著落差，限制其實務應用。
為回應此困境，本研究設計一套模組化的 AI 輔助論文審查系統，以提升生成評論的準確性、專業性與一致性。整體流程涵蓋三大設計理念：第一，透過記憶模組進行反思學習，記錄過往錯誤模式並轉化為泛化建議，使模型在未來任務中能主動修正偏誤，提升判斷穩定性與可追溯性。第二，於生成階段引入檢索增強生成（Retrieval-Augmented Generation, RAG）技術，針對論文主題動態檢索跨領域知識，作為外部依據補充，以增強評論內容的事實性與深度。第三，透過多代理系統（Multi-Agent System, MAS）將複雜的審查任務分解為多個具體面向，分別由不同功能的代理人處理，包括可讀性、方法嚴謹性與學術貢獻等，最後由統整代理人彙整各項輸出，生成結構化且具決策參考價值的審查意見。該系統不僅提升回饋品質，亦具備模組化與擴展彈性，能靈活對應不同審查需求與應用場景。
本研究使用 ECNU-SEA/SEA-E 作為基線，在測試集的 F1 效能為 0.700，單獨引入記憶模組後，F1 分數可由 0.700 提升至 0.771，主因在於其有效提高正確拒絕樣本的比例（True Positive, TP），且並未犧牲正確接受的準確性（TN）；單純使用多代理架構即可將效能提升至 0.750；進一步結合 QLoRA 微調各代理人後可達 0.786，若再整合記憶模組，則最高可達 0.801。證實記憶模組與代理系統具備加乘效果，在精度與穩定性上展現優勢，也具備良好的擴展潛力與模組解釋性。;With the rapid growth of global academic output, traditional peer review faces mounting challenges such as reviewer shortages and inconsistent quality. While large language models (LLMs) offer initial support for automating this process, they still fall short of human experts in accuracy, interpretability, and bias control, limiting their practical adoption.
To overcome these issues, we propose a modular AI-assisted review system designed to enhance the accuracy, consistency, and professionalism of automated reviews. The system integrates three key components: (1) a Memory Module for reflective learning, which records past errors and improves decision stability; (2) Retrieval-Augmented Generation (RAG) to incorporate cross-domain factual knowledge; and (3) a Multi-Agent System (MAS) that delegates subtasks—such as readability, methodology, and contribution—to specialized agents, with a final agent consolidating the outputs.
Using ECNU-SEA/SEA-E as the base model (F1 = 0.700), adding the Memory Module boosts performance to 0.771 by significantly improving true positive (TP) rates. The MAS alone achieves 0.750, while fine-tuning each agent with QLoRA increases it to 0.786. Combining MAS and Memory further lifts performance to 0.801, demonstrating the system’s synergy, scalability, and robustness. |
