基於參數生成網絡的遷移學習進行情感分析和歌手命名識別

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：89

、訪客IP：3.144.40.46

姓名

曾筱雯(Hsiao-Wen Tseng) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於參數生成網絡的遷移學習進行情感分析和歌手命名識別
(Parameter Generation Network Based Transfer Learning for Aspect-Based Sentiment Analysis and Singer Name Recognition)

相關論文

★ 行程邀約郵件的辨識與不規則時間擷取之研究	★ NCUFree校園無線網路平台設計及應用服務開發
★ 網際網路半結構性資料擷取系統之設計與實作	★ 非簡單瀏覽路徑之探勘與應用
★ 遞增資料關聯式規則探勘之改進	★ 應用卡方獨立性檢定於關連式分類問題
★ 中文資料擷取系統之設計與研究	★ 非數值型資料視覺化與兼具主客觀的分群
★ 關聯性字組在文件摘要上的探討	★ 淨化網頁：網頁區塊化以及資料區域擷取
★ 問題答覆系統使用語句分類排序方式之設計與研究	★ 時序資料庫中緊密頻繁連續事件型樣之有效探勘
★ 星狀座標之軸排列於群聚視覺化之應用	★ 由瀏覽歷程自動產生網頁抓取程式之研究
★ 動態網頁之樣版與資料分析研究	★ 同性質網頁資料整合之自動化研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

隨著科技的日新月異，在網路上表達自己的看法變得更加便利。因此對某個領域有興趣時，可以偵測網路聲量，進行各種分析。不過單純只討論提及次數，難以得到正確得評價，因為有可能這則評論真正的意見不是針對提及的人物。且使用者在社群媒體上的敘述較為口語，較不依循正規的文法表達方式，加上收集新的領域資料後，需要花大量的時間、金錢進行標記，因此本篇論文希望從這些資料中找出正確的意見目標，且使用標記資料訓練的模型，幫助新的領域資料進行標記。

因此使用遷移式學習 (Transformer Learning) 的方向設計模型架構，以多任務學習 (Multi-task Learning) 的方式進行中文歌手的辨識 (Named Entity Recognition, NER) 和基於面向的情感分析 (Aspect-Based Sentiment Analysis, ABSA) 的任務。我們應用參數生成網路結合梯度反轉層 (Gradient Adversarial Layer, GRL) 來建立模型。並使用 Tie/Break 進行標記，以此提升中文斷詞的準確度。透過動態調節權重的機制 (Dynamic Weight Average, DWA) ，依據每個任務的損失變化率來調整任務權重。

實驗結果顯示，我們的擴展參數生成網路模型 (Extended Parameter Generation Network, E-PGN)，在僅考慮 NER 任務時， F1 可以達到 90\% ，和 IBHB 效能 86% 相比，有所改善，加入 ABSA 任務後，平均 F1 能夠達到 78% ，和 IBHB 效能相差了 22% ，明顯的大幅成長。

摘要(英)

With the rapid development of technology, it has become more convenient to express opinions on the Internet.
Therefore, when you are interested in a certain field, you can detect the number of network sounds and perform various analyses.
However, it is difficult to get a correct evaluation only by discussing the number of mentions, because the true opinion of this review is probably not aimed at the person mentioned.
In addition, users’ narratives on social media are more colloquial and do not follow formal grammatical expressions.
After collecting new domain data, it takes a lot of time and money to label.
Therefore, this paper hopes to find the correct opinion target from these data, and use the model trained by the labeled data to help the new domain data to be labeled.

Therefore, we use the direction of Transformer Learning to design the model architecture, and perform Named Entity Recognition (NER) and Aspect-Based Sentiment Analysis (ABSA) tasks in the way of Multi-task Learning.
We use the Parameter Generation Network (PGN) combined with the Gradient Reversal layer (GRL) to build the model.
And use Tie/Break for labeling to improve the accuracy of Chinese word segmentation.
Through Dynamic Weight Average (DWA) describe the loss change rate of each task to adjust the task weight.

Experimental results show that when only NER tasks are considered, our Extended Parameter Generation Network (E-PGN) can reach 90%, which is 86% more efficient than IBHB.
After joining the ABSA task, the average F1 score can reach 78%, which is 22% different from the performance of IBHB, which is a significant improvement.

關鍵字(中)

★ 參數生成網路
★ 梯度反轉層
★ 命名實體辨識
★ 目標情緒分析

關鍵字(英)

★ Parameter Generation Network
★ Gradient Adversarial Layer
★ Named Entity Recognition
★ Aspect-Based Sentiment Analysis

論文目次

中文摘要 i
Abstract ii
目錄 iii
圖目錄 iv
表目錄 v
一、簡介 1
二、相關研究 4
2.1 命名實體辨識(Named Entity Recognition,NER) 4
2.2 基於方面情感分析(Aspect Based Sentiment Analysis, ABSA) 4
2.3 遷移式學習(Transfer Learning) 5
2.4 參數生成網路(Parameter Generation Network, PGN) 8
2.5 神經網路的域對抗訓練(Domain-Adversarial Training of Neural Networks, DANN) 9
三、模型架構 11
四、實驗 16
4.1 資料分析 16
4.1.1 歌手文章 16
4.1.2 新聞 17
4.1.3 家用電器評論 18
4.2 標記策略 18
4.3 效能評估與參數設定 19
4.4 模型效能 20
4.4.1 來源域為家用電器評論 20
4.4.2 來源域為新聞 24
五、結論 28
參考文獻 29

參考文獻

[1]Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich.Gradnorm: Gradient normalization for adaptive loss balancing in deep mul-titask networks. InICML, 2018.

[2]Wei-Cheng Chiu. Joint learning of aspect-level sentiment analysis and singernamed recognition from social networks. InInternational Conference on Tech-nologies and Applications of Artificial Intelligence (TAAI), 2020.

[3]Wei-Cheng Chiu. Joint learning of aspect-level sentiment analysis and singernamed recognition from social networks. InInternational Conference on Tech-nologies and Applications of Artificial Intelligence (TAAI), 2020.

[4]Chien-Lung Chou, Chia-Hui Chang, and Ya-Yun Huang. Boosted web namedentity recognition via tri-training.ACM Trans. Asian Low-Resour. Lang. Inf.Process., 16(2), October 2016.

[5]Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, KorayKavukcuoglu, and Pavel Kuksa. Natural language processing (almost) fromscratch.J. Mach. Learn. Res., 12(null):24932537, November 2011.

[6]Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu. Boosting for trans-fer learning. InProceedings of the 24th International Conference on MachineLearning, ICML ’07, page 193200, New York, NY, USA, 2007. Association forComputing Machinery.

[7]J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InNAACL-HLT, 2019.

[8]Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation bybackpropagation. InProceedings of the 32nd International Conference on In-ternational Conference on Machine Learning - Volume 37, ICML’15, page11801189. JMLR.org, 2015.

[9]Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. An interactivemulti-task learning network for end-to-end aspect-based sentiment analysis. InProceedings of the 57th Annual Meeting of the Association for ComputationalLinguistics, pages 504–515, 2019

[10]Mengting Hu, Yike Wu, Shiwan Zhao, Honglei Guo, Renhong Cheng, and ZhongSu. Domain-invariant feature distillation for cross-domain sentiment classifica-tion. InProceedings of the 2019 Conference on Empirical Methods in NaturalLanguage Processing and the 9th International Joint Conference on NaturalLanguage Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November3-7, 2019, pages 5558–5567, 2019.

[11]Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequencetagging.CoRR, abs/1508.01991, 2015.

[12]Chen Jia, Xiaobo Liang, and Yue Zhang. Cross-domain NER using cross-domain language modeling. InProceedings of the 57th Annual Meeting of theAssociation for Computational Linguistics, pages 2464–2474. Association forComputational Linguistics, July 2019.

[13]John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditionalrandom fields: Probabilistic models for segmenting and labeling sequence data.InProceedings of the Eighteenth International Conference on Machine Learning,ICML ’01, pages 282–289. Morgan Kaufmann Publishers Inc., 2001.

[14]Peng-Hsuan Li, Tsu-Jui Fu, and Wei-Yun Ma. Why attention? analyze bilstmdeficiency and its remedies in the case of ner. InAAAI, 2020.

[15]Zheng Li, Xin Li, Ying Wei, Lidong Bing, Y. Zhang, and Qiang Yang. Trans-ferable end-to-end aspect-based sentiment analysis with selective adversariallearning, 2019.

[16]Shikun Liu, Edward Johns, and A. Davison. End-to-end multi-task learningwith attention.2019 IEEE/CVF Conference on Computer Vision and PatternRecognition (CVPR), pages 1871–1880, 2019.

[17]Andrew McCallum, Dayne Freitag, and Fernando C. N. Pereira. Maximum en-tropy markov models for information extraction and segmentation. InProceed-ings of the Seventeenth International Conference on Machine Learning, ICML’00, page 591598. Morgan Kaufmann Publishers Inc., 2000.

[18]Lilyana Mihalkova, Tuyen Huynh, and Raymond J. Mooney. Mapping and revis-ing markov logic networks for transfer learning. InProceedings of the 22nd Na-tional Conference on Artificial Intelligence - Volume 1, AAAI’07, page 608614.AAAI Press, 2007.

[19]Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean.Distributed representations of words and phrases and their compositionality. InProceedings of the 26th International Conference on Neural Information Pro-cessing Systems - Volume 2, NIPS’13, page 31113119. Curran Associates Inc.,2013.

[20]Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok, and Qiang Yang. Domainadaptation via transfer component analysis.IEEE Transactions on Neural Net-works, pages 199–210, 2011.

[21]Sinno Jialin Pan and Qiang Yang. A survey on transfer learning.IEEE Trans-actions on Knowledge and Data Engineering, 22(10):1345–1359, 2010.

[22]Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove:Global vectors for word representation. InEMNLP, volume 14, pages 1532–1543, 2014.

[23]Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, ChristopherClark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word represen-tations, 2018. cite arxiv:1802.05365Comment: NAACL 2018. Originally postedto openreview 27 Oct 2017. v2 updated for NAACL camera ready.

[24]Lawrence R. Rabiner.A Tutorial on Hidden Markov Models and Selected Ap-plications in Speech Recognition, page 267296. Morgan Kaufmann PublishersInc., 1990.

[25]Alec Radford and Ilya Sutskever. Improving language understanding by gener-ative pre-training. Inarxiv, 2018.

[26]Kumar Ravi and V. Ravi. A survey on opinion mining and sentiment analysis:Tasks, approaches and applications.Knowl. Based Syst., 89:14–46, 2015.

[27]K. Schouten and F. Frasincar. Survey on aspect-level sentiment analysis.IEEETransactions on Knowledge and Data Engineering, 28:813–830, 2016.

[28]Jingbo Shang, Liyuan Liu, Xiang Ren, X. Gu, Teng Ren, and Jiawei Han.Learning named entity tagger using domain-specific dictionary. InEMNLP,2018.

[29]Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. Effective lstms fortarget-dependent sentiment classification. InProceedings of COLING 2016, the26th International Conference on Computational Linguistics: Technical Papers,pages 3298–3307, 2016.

[30]Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. Attention-basedLSTM for aspect-level sentiment classification. InProceedings of the 2016 Con-ference on Empirical Methods in Natural Language Processing, pages 606–615.Association for Computational Linguistics, November 2016.

[31]Hu Xu, Bing Liu, Lei Shu, and Philip S. Yu. Bert post-training for reviewreading comprehension and aspect-based sentiment analysis. InNAACL, 2019.

[32]Wei Xue and Tao Li. Aspect based sentiment analysis with gated convolutionalnetworks. InProceedings of the 56th Annual Meeting of the Association forComputational Linguistics (Volume 1: Long Papers), pages 2514–2523. Associ-ation for Computational Linguistics, July 2018.

[33]Kai Zhang, Hefu Zhang, Qi Liu, Hongke Zhao, Hengshu Zhu, and Enhong Chen.Interactive attention transfer network for cross-domain sentiment classification.Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):5773–5780, Jul. 2019.

[34]Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, HengshuZhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning.Proceedings of the Institute of Radio Engineers, 109(1):43–76, January 2021.

指導教授

張嘉惠(Chia-Hui Chang)

審核日期

2021-8-17

推文