運用異質圖注意力網路於中文醫療答案擷取式摘要

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：27

、訪客IP：18.227.13.169

姓名

田高源(Kao-Yuan Tien) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

運用異質圖注意力網路於中文醫療答案擷取式摘要
(Heterogeneous Graph Attention Networks for Extractive Summarization of Chinese Medical Answers)

相關論文

★ 多重嵌入增強式門控圖序列神經網路之中文健康照護命名實體辨識	★ 基於腦電圖小波分析之中風病人癲癇偵測研究
★ 基於條件式生成對抗網路之資料擴增於思覺失調症自動判別	★ 標籤圖卷積增強式超圖注意力網路之中文健康照護文本多重分類
★ 運用合成器混合注意力改善BERT模型於科學語言編輯	★ 強化領域知識語言模型於中文醫療問題意圖分類
★ 管道式語言轉譯器之中文健康照護開放資訊擷取	★ 運用句嵌入向量重排序器增進中文醫療問答系統效能
★ 利用雙重註釋編碼器於中文健康照護實體連結	★ 聯合詞性與局部語境於中文健康照護實體關係擷取
★ 學習使用者意圖於中文醫療問題生成式摘要	★ 標籤強化超圖注意力網路模型於精神疾病文本多標籤分類

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-10-12以後開放)

摘要(中)

檢索式醫療問答系統藉由問題與答案的配對排序，回覆使用者的醫療相關問題。然而，返回的相關資訊通常多樣複雜，對於尋找特定資訊的使用者來說，這些答案通常需要花費時間閱讀與理解。本研究專注於中文醫療答案摘要問題，藉由文本摘要技術，將冗長複雜的相關資訊，擷取成簡潔易於理解的答案。我們提出一個基於異質圖注意力網路的擷取式摘要模型 (Heterogeneous Graph Attention Networks for Extractive Summarization, HGATSUM)，用於檢索式中文醫療問答系統。首先，我們將醫療問題和答案對建構成異質圖，圖節點包含問題、答案以及醫療實體，節點間關係做為邊，包含1) 答案句子間基於修辭結構理論的依賴關係; 2) 問題與答案句子間的相似關係; 以及3) 醫療實體和問題或答案句子間的提及關係。然後，經由圖注意力網路來學習異質圖的節點表示。最後，將答案句子的圖節點表示與相關性特徵結合後，進行答案中的句子選擇與組合，形成最終輸出摘要答案。
由於缺乏公開的評測資料集，我們建置了一個中文醫療答案擷取式摘要任務的資料集 (Med-AnsSum)，包含469筆醫療問題，以及這些問題藉由檢索系統返回的問答配對共有3,314筆，每筆皆人工標記擷取摘要答案。藉由實驗與效能評估得知，我們提出的模型HGATSUM在資料集Med-AnsSum上的ROUGE (1/2/L) 分數表現 (82.08/78.66/81.60)，皆優於其他相關模型(BERTSUMEXT, MATCHSUM, AREDSUM以及Bert-QSBUM)，人工評估進一步驗證我們提出的HGATSUM模型在中文醫療答案擷取式摘要上有良好的表現。

摘要(英)

Information retrieval-based medical question-answering systems usually return relevant answers to a user’s question in a ranked list. However, retrieved results may contain complex and diverse information that hinders users from meeting their specific question intents easily. Therefore, this study focuses on developing extractive summarization techniques for Chinese medical answers. We propose a model called HGATSUM (Heterogeneous Graph Attention Networks for Summarization). First, we construct a heterogeneous graph comprised of nodes in terms of questions, answer sentences, and medical entities and their relationships as edges, including 1) dependency relationships based on Rhetorical Structure Theory (RST) among answer sentences; 2) similarity relationships between questions and answer sentences; and 3) mention relationships between entities and question/answer sentences. Then, Graph Attention Networks are used to learn feature representations of heterogeneous graph nodes. Finally, we combine the graph features of answer sentences with relevancy to the posed question for selecting and assembling partial sentences as an extracted summary.
Due to a lack of publicly released benchmark data for medical answer summarization, we constructed a dataset called Med-AnsSum for the extractive summarization task of Chinese medical answers. This dataset contains 3,314 question-answer pairs across 469 distinct medical questions returned by the medical question-answering system, each was manually annotated to obtain an extractive answer summary. Based on experiments and performance evaluations, our proposed HGATSUM model outperforms previous models (i.e., BERTSUMEXT, MATCHSUM, AREDSUM, and Bert-QSBUM) on the Med-AnsSum dataset, achieving the best ROUGE-(1/2/L) scores of 82.08/78.66/81.60. The human evaluation also confirmed that our model is an effective method for Chinese medical answer summarization.

關鍵字(中)

★ 擷取式摘要
★ 異質圖
★ 圖注意力網路

關鍵字(英)

★ extractive summarization
★ heterogeneous graph
★ graph attention networks

論文目次

摘要 i
ABSTRACT ii
誌謝 iii
目錄 iv
表目錄 vi
圖目錄 vii
第一章緒論 1
1-1 研究背景 1
1-2 研究動機 3
1-3 研究目的 4
1-4 章節概要 6
第二章相關研究 7
2-1 摘要模型 7
2-1-1 非監督式學習方法 7
2-1-2 監督式學習方法 8
2-2 資料集 15
第三章研究方法 18
3-1 系統架構 18
3-2 異質圖建構 20
3-3 圖節點初始化 24
3-4 異質圖編碼器 27
3-5 摘要層 29
第四章實驗與效能評估 30
4-1 資料集建置 30
4-2 效能指標 33
4-3 實驗設定 37
4-4 模型比較 38
4-5 消融實驗 40
4-6 嵌入向量分析 42
4-7 異質圖編碼器分析 43
4-8 人工評估 44
4-9 大型語言模型比較 49
第五章研究結論 53
5-1 結論 53
5-2 研究限制 54
5-3 未來工作 55
參考文獻 56

參考文獻

Bi, K., Jha, R., Croft, B., & Celikyilmaz, A. (2021, April). AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization.Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume Online.
Cui, Y., Che, W., Liu, T., Qin, B., Wang, S., & Hu, G. (2020, November). Revisiting Pre-Trained Models for Chinese Natural Language Processing.Findings of the Association for Computational Linguistics: EMNLP 2020 Online.
Cui, Y., Che, W., Liu, T., Qin, B., & Yang, Z. (2021). Pre-Training With Whole Word Masking for Chinese BERT. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 3504-3514. https://doi.org/10.1109/TASLP.2021.3124365
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019, June). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) Minneapolis, Minnesota.
Efron, B., & Tibshirani, R. J. (1994). An introduction to the bootstrap. CRC press.
Hermann, K. M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015). Teaching machines to read and comprehend. Advances in neural information processing systems, 28.
Hung, S.-S., Huang, H.-H., & Chen, H.-H. (2020, July). A Complete Shift-Reduce Chinese Discourse Parser with Robust Dynamic Oracle.Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics Online.
Jia, R., Cao, Y., Tang, H., Fang, F., Cao, C., & Wang, S. (2020, November). Neural Extractive Summarization with Hierarchical Attentive Heterogeneous Graph Network.Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) Online.
Lee, L. H., & Lu, Y. (2021). Multiple Embeddings Enhanced Multi-Graph Neural Networks for Chinese Healthcare Named Entity Recognition. IEEE Journal of Biomedical and Health Informatics, 25(7), 2801-2810. https://doi.org/10.1109/JBHI.2020.3048700
Lin, C.-Y. (2004, July). ROUGE: A Package for Automatic Evaluation of Summaries.Text Summarization Branches Out Barcelona, Spain.
Liu, Y., & Lapata, M. (2019, November). Text Summarization with Pretrained Encoders.Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) Hong Kong, China.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Mihalcea, R., & Tarau, P. (2004, July). TextRank: Bringing Order into Text.Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing Barcelona, Spain.
Nallapati, R., Zhai, F., & Zhou, B. (2017). SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10958
Narayan, S., Cohen, S. B., & Lapata, M. (2018, June). Ranking Sentences for Extractive Summarization with Reinforcement Learning.Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) New Orleans, Louisiana.
Petar, V., Guillem, C., Arantxa, C., Adriana, R., Pietro, L., & Yoshua, B. (2018). Graph Attention Networks International Conference on Learning Representations, https://openreview.net/forum?id=rJXMpikCZ ,
Sandhaus, E. (2008). The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, 6(12).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Wang, H., Liu, C., Xi, N., Qiang, Z., Zhao, S., Qin, B., & Liu, T. (2023). Huatuo: Tuning llama model with chinese medical knowledge. arXiv preprint arXiv:2304.06975.
Zhao, M., Yan, S., Liu, B., Zhong, X., Hao, Q., Chen, H., Niu, D., Long, B., & Guo, W. (2021). QBSUM: A large-scale query-based document summarization dataset from real-world applications. Computer Speech & Language, 66, 101166. https://doi.org/https://doi.org/10.1016/j.csl.2020.101166
Zhong, M., Liu, P., Chen, Y., Wang, D., Qiu, X., & Huang, X. (2020, July). Extractive Summarization as Text Matching.Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics Online.
Zhou, Q., Yang, N., Wei, F., Huang, S., Zhou, M., & Zhao, T. (2018, July). Neural Document Summarization by Jointly Learning to Score and Select Sentences.Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Melbourne, Australia.

指導教授

李龍豪(Lung-Hao Lee)

審核日期

2023-10-13

推文