基於類代理注意力特徵融合模型的聯合 實體關係抽取方法

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：171

、訪客IP：3.146.105.212

姓名

葉航愷(Hang-Kai Ye) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

基於類代理注意力特徵融合模型的聯合實體關係抽取方法
(Agent-like Attention Relation Feature Fusion Model for Joint Entity Relation Extraction Method)

相關論文

★ 基於適應性徑向基神經網路與非奇異快速終端滑模控制結合線上延遲估測器應用於二軸機械臂運動軌跡精確控制	★ 新型三維光學影像量測系統之設計與控制
★ 新型雙紐線軌跡設計與進階控制實現壓電平台快速與精確定位	★ 基於深度座標卷積與自動編碼器給予行人實時路徑及終點位置精確預測
★ 修正式雙紐線軌跡結合自適應積分終端滑動模態控制與逆模型遲滯補償實現壓電平台精確追蹤	★ 以粒子群最佳化-倒傳遞類神經網路-比例積分微分控制器和影像金字塔轉換融合方法實現三維光學顯微影像系統
★ 以局部熵亂度分布與模板匹配方法結合自適應ORB特徵提取達成多影像精確拼接	★ 低扭矩機械手臂機構開發與脈寬調變進階控制器設計
★ 使用時域門控與梅森增益公式構建四埠夾具的散射參數表徵	★ 通過強化學習與積分滑模動量觀測器實現機器手臂的強健近佳PD控制策略
★ 新型修正式柵欄軌跡結合擴增狀態估測滑模回授與多自由度Bouc-Wen遲滯前饋補償控制器給予壓電平台快速精確追蹤	★ 結合點雲密度熵計算方法和運動回復結構在虛幻引擎中進行影像三維點雲模型及渲染重建

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2029-7-1以後開放)

摘要(中)

關係抽取是信息抽取任務中非常重要的一項子任務，核心目的是從句子中尋找存在關係的實體對，並為實體對匹配對應的關係類型。本論文在對關係抽取任務的研究中提出了應用於關係抽取任務的一種類代理特徵融合模型。該方法在一種級聯二進制標註的關係抽取框架基礎上，加入了一個類代理注意力特徵融合模塊，以提高模型對句子中關係實體的抽取效率。該模塊通過深度學習與訓練一個包含句子上下文語義信息與句子類別信息的代理向量，接著模塊會對代理向量中的有效信息進行再次提煉，幫助模型增強訓練數據中提取關係實體和關係類別中隱含的語義信息能力。藉由實驗結果證實，類代理注意特徵融合模型的關係抽取性能相比CasRel有明顯提升，因此，本論文所提出的模型可以有效地提高關係三元組的抽取效率。

摘要(英)

Relation extraction is a very important subtask in the information extraction task. The core purpose is to find entity pairs with relations in sentences and match the corresponding relation types for the entity pairs. In the study of relation extraction tasks, this thesis proposes an agent-like attention feature fusion model for relation extraction tasks. Based on a cascade binary tagging relation extraction framework, this method adds an agent-like attention feature fusion module to improve the model’s extraction efficiency of relational entities in sentences. This module trains an agent vector containing sentence context semantic information and sentence category information. Then, the module will further refine the effective information in the agent vector to help the model enhance its ability to extract implicit semantic information in relational entities and relation categories from training data. Experimental results show that the relation extraction performance of the agent-like attention feature fusion model is significantly improved compared with CasRel. Therefore, the model proposed in this thesis effectively improves the extraction efficiency of relational triples.

關鍵字(中)

★ 關係抽取任務
★ 類代理特徵融合模型
★ 深度學習
★ 抽取效率

關鍵字(英)

★ relation extraction task
★ agent-like attention feature fusion model
★ deep learning
★ extraction efficiency

論文目次

摘要 i
ABSTRACT ii
誌謝 iii
Table of Content v
List of Figures vii
List of Tables viii
Explanation of Symbols ix
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Relation Exaction 3
1.2.1 Relation Triple 3
1.2.2 Overlapping Triple Problem 3
1.2.3 Pipeline-based Relation Extraction method 5
1.2.4 Joint entity Relation Extraction method 6
1.3 Literature Survey 7
1.4 Contribution 13
1.5 Thesis Organization 15
Chapter 2 Preliminaries 17
2.1 Overlapping Triple Classification 17
2.2 BERT 18
2.3 Attention Mechanism 24
2.4 Agent Attention Mechanism 30
2.4.1 Computational Complexity 30
2.4.2 Agent Attention 32
Chapter 3 Relation Entity Extraction Processes 35
3.1 Relation Entity Pair Extraction 35
3.1.1 Subject Tagger 36
3.1.2 Relation-specific Object Tagger 38
3.2 Overall Objective Optimizer Function 41
Chapter 4 Agent-like Attention Feature Fuse Model 43
4.1 Vector Encoding Module 44
4.1.1 Initial Context Vector 44
4.1.2 Initial Relation Vector 46
4.2 Agent-like Attention Layer 48
4.2.1 Agent Parameters Calculation 48
4.2.2 Agent-like Attention feature fuse Calculation 50
4.3 Sequence Entity Tagger 53
4.4 Loss Function 58
4.5 Implement Detail 60
Chapter 5 Experiments 61
5.1 Datasets 61
5.2 Evaluation Metrics 63
5.3 Experimental Result 67
5.3.1 Comparison Method Introduction 68
5.3.2 Main Result 70
5.3.3 Extraction Results on Different Types of Overlapping Triple 71
5.3.4 Extraction Results on Number of Relation Triple 73
5.3.5 Comparing Results of Training Time 75
5.4 Case Verification 76
Chapter 6 Conclusions 78
Reference 79

參考文獻

[1] D. Zelenko, C. Aone, and A. Richardella, “Kernel Methods for Relation Extraction,” Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 71-78, 2002.
[2] M. Miwa and M. Bansal, “End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures,” Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1105-1116, 2016.
[3] Y. Lin, S. Shen, Z. Liu, H. Luan, and M. Sun, “Neural Relation Extraction with Selective Attention over Instances,” Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2124-2133, 2016.
[4] C. Alt, M. Hübner, and L. Hennig, “Improving Relation Extraction by Pre-trained Language Representations,” ArXiv abs/1906.03088, 2019.
[5] Z. Wei, J. Su, Y. Wang, Y. Tian, and Y. Chang, “A Novel Cascade Binary Tagging Framework for Relational Triple Extraction,” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1476-1488, 2020.
[6] S. Zheng, F. Wang, H. Bao, Y. Hao, P. Zhou, and B. Xu, “Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme,” Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1227-1236, 2017.
[7] Y. Wang, B. Yu, Y. Zhang, T. Liu, H. Zhu, and L. Sun, “TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking,” Proceedings of the 28th International Conference on Computational Linguistics, pp. 1572-1582, 2020.
[8] Y. Wang, C. Sun, Y. Wu, H. Zhou, L. Li, and J. Yan, “UniRE: A Unified Label Space for Entity Relation Extraction,” Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 220-231, 2021.
[9] D. Ye, Y. Lin, P. Li, and M. Sun, “Packed Levitated Marker for Entity and Relation Extraction,” Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4904-4917, 2022.
[10] J. Zhao, W. Zhan, X. Zhao, Q. Zhang, T. Gui, Z. Wei, J. Wang, M. Peng, and M. Sun, “RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction,” Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6680-6691, 2023.
[11] R. Zhang, Y. Li, and L. Zou, “A Novel Table-to-Graph Generation Approach for Document-Level Joint Entity and Relation Extraction,” Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10853-10865, 2023.
[12] Z. Zhong and D. Chen, “A Frustratingly Easy Approach for Entity and Relation Extraction,” Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 50-61, 2021.
[13] D. Rumelhart, G. Hinton, and R. Williams, “Learning representations by back-propagating errors,” Nature 323, pp. 533-536, 1986.
[14] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[15] Y. LeCun, “Generalization and network design strategies,” 1989.
[16] J. Lafferty, A. Callum and F. Pereira, “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data,” International Conference on Machine Learning, 2001.
[17] C. Cortes and V. Vapnik, “V. Support-vector networks,” Mach Learn 20, pp. 273-297, 1995.
[18] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin, “Attention is All you Need,” Neural Information Processing Systems, 2017.
[19] J. West, D. Ventura, and S. Warnick, “Spring Research Presentation: A Theoretical Foundation for Inductive Transfer,” Brigham Young University, College of Physical and Mathematical Sciences, pp. 32, 2007.
[20] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” North American Chapter of the Association for Computational Linguistics, 2019.
[21] D. Han, T. Ye, Y. Han, Z. Xia, S. Song, and G. Huang, “Agent Attention: On the Integration of Softmax and Linear Attention,” ArXiv abs/2312.08874, 2023.
[22] S. Riedel, L. Yao, A. McCallum, “Modeling relations and their mentions without labeled text,” Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 148-163, 2010.
[23] C. Gardent, A. Shimorina, S. Narayan, and L. Perez-Beltrachini, “Creating training corpora for nlg micro-planners,” Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 179-188, 2017.
[24] X. Zeng, D. Zeng, S. He, K. Liu, and J. Zhao, “Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism,” Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 506-514, 2018.
[25] T. Fu, P. Li, and W. Ma, “GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1409-1418, 2019.
[26] J. Cheng, T. Zhang, S. Zhang, H. Ren and G. Yu, “A Cascade Dual-Decoder Model for Joint Entity and Relation Extraction,” IEEE Transactions on Emerging Topics in Computational Intelligence, pp. 1-13, 2024.
[27] M. Hearst, “Automatic Acquisition of Hyponyms from Large Text Corpora,” Proceedings of the 14th International Conference on Computational Linguistics, 1992.
[28] I. Sutskever, O. Vinyals, and Q. Le, “Sequence to sequence learning with neural networks,” Proceedings of the 27th International Conference on Neural Information Processing Systems, pp. 3104-3112, 2014.
[29] L. Pratt, “Discriminability-based transfer between neural networks,” Proceedings of the 5th Neural Information Processing Systems, pp. 204-211, 1993.
[30] S. Stephen, “Selecting and interpreting measures of thematic classification accuracy,” Proceedings of Remote Sensing of Environment, pp. 77-89, 1997.
[31] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.

指導教授

吳俊緯(Jim-Wei Wu)

審核日期

2024-7-23

推文