結合Selective Mechanism與多向注意力機制應用於自動文本摘要之研究

NCUIR > School of Management at National Central University > Graduate Institute of Information Management > Electronic Thesis & Dissertation > Item 987654321/86637

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/86637

Title:	結合Selective Mechanism與多向注意力機制應用於自動文本摘要之研究
Authors:	董子瑄;Tung, Tzu-Hsuan
Contributors:	資訊管理學系
Keywords:	Transformer;Selective Mechanism;自注意力機制;萃取式摘要;中文文本摘要;Transformer;Selective mechanism;Self-attention;Abstractive summarization;Chinese summarization
Date:	2021-08-02
Issue Date:	2021-12-07 13:02:48 (UTC+8)
Publisher:	國立中央大學
Abstract:	文本摘要任務的目的在於將原始文本以精簡的文字重新呈現，同時要保留重點且不失原文語意。本研究結合Selective Mechanism與Transformer模型中的多向注意力機制以提升萃取式摘要模型的生成摘要品質，透過一個可訓練的Selective Gate Network對Transformer編碼器的多向注意力輸出進行過濾，產生二次潛在語意向量，以達到精煉的效果，其目的在於以過濾的方式，除去次要的資訊，萃取出應保留在摘要中的重點資訊，並使用二次潛在語意向量進行解碼，來產生更好的摘要。本研究並將此模型應用於中文文本摘要生成上，以ROUGE值做為評估指標，實驗結果顯示此模型在ROUGE-1、ROUGE-2、ROUGE-L都能超越Baseline模型，在Word-based ROUGE上提升約7.3~12.7%，在Character-based ROUGE上提升約4.9~7.9%，此外搭配Word to Character的斷詞方法並擴大編碼器更可以大幅提升各項ROUGE指標，在Word-based ROUGE可再提升20.4~41.8%，Character-based ROUGE可再提升約21.5~31.1%。;Text summarization task aims to represent the original article in condensed text, while retaining the key points and the original semantics. This research combines selective mechanism with multi-head attention to improve the generated summary quality of the abstractive summarization model. A trainable selective gate network is used to filter the multi-head attention outputs in the Transformer encoder, which can select important information and discard unimportant information, and finally construct second level representation. The second level representation is a tailored sentence representation, which can be decoded into a better summary. This model is applied to Chinese text summarization task, and the evaluation metric is ROUGE score. The experiment result shows that the model performance exceed the baseline by 7.3 to 12.7% on word-based ROUGE, and 4.9 to 7.9% on character-based ROUGE. Moreover, with word to character tokenization and larger vocabulary banks can significantly improve the performance. In word-based ROUGE, it can increase by 20.4 to 41.8%, and character-based ROUGE can increase by 21.5 to 31.1%.
Appears in Collections:	[Graduate Institute of Information Management] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	104	View/Open

社群 sharing

Loading...