應用角色感知於深度神經網路架構之對話行為分類

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：16

、訪客IP：18.221.41.214

姓名

王睿揚(Jui-Yang Wang) 查詢紙本館藏

畢業系所

軟體工程研究所

論文名稱

應用角色感知於深度神經網路架構之對話行為分類
(Dialog act Classification with Role awareness in DNN Framework)

相關論文

★ A Real-time Embedding Increasing for Session-based Recommendation with Graph Neural Networks	★ 基於主診斷的訓練目標修改用於出院病摘之十代國際疾病分類任務
★ 混合式心臟疾病危險因子與其病程辨識於電子病歷之研究	★ 基於 PowerDesigner 規範需求分析產出之快速導入方法
★ 社群論壇之問題檢索	★ 非監督式歷史文本事件類型識別──以《明實錄》中之衛所事件為例
★ 應用自然語言處理技術分析文學小說角色之關係：以互動視覺化呈現	★ 基於生醫文本擷取功能性層級之生物學表徵語言敘述：由主成分分析發想之K近鄰算法
★ 基於分類系統建立文章表示向量應用於跨語言線上百科連結	★ Code-Mixing Language Model for Sentiment Analysis in Code-Mixing Data
★ 藉由加入多重語音辨識結果來改善對話狀態追蹤	★ 主動式學習之古漢語斷詞
★ 對話系統應用於中文線上客服助理:以電信領域為例	★ 應用遞歸神經網路於適當的時機回答問題
★ 使用多任務學習改善使用者意圖分類	★ 使用轉移學習來改進針對命名實體音譯的樞軸語言方法

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

在自然語言領域中，對話機器人應用日益發展迅速，其中需要克服問題之一在於自然語言理解，知道使用者在詢問何種問題及判斷文字間隱藏的資訊，對於使機器了解使用者的問題意圖是相當重要。後續的應用例如對話管理以及如何產生相應的答案皆會需要根據意圖理解來做延伸，因此如何達到更好的辨識率將是一大挑戰。
本研究主要針對對話資料訓練深度學習模型並預測對話行為，運用各種神經網路來解決此困難，並比較其之間的差異，同時引入角色資訊於模型中，找出能針對中文短句子特性能夠有效處理的模型，另外，在模型加入預訓練詞向量，能夠更有效處理中文未知詞，以減少錯誤辨識的可能。最終，本論文比較多種深度學習模型並引入角色資訊來辨識對話行為，相比一般的神經網路模型在電信領域對話資料集提升將近1.2%。

摘要(英)

In the field of natural language processing, the application of dialogue robot is growing rapidly. One of the problems that need to be overcome in the field is natural language understanding. Knowing what kind of question the user is asking and judging the hidden information between the words, and the intention of making the machine understand the problem of the user is very important. Also the follow-up parts such as dialog management and how to produce the corresponding answer will need to be interpreted according to intent to do, so how to catch a better recognition rate will be a big challenge.
In this study, we mainly train the deep learning model for dialogue data and predict the dialogue act. We use various neural networks to solve this problem and compare the differences. At the same time, we introduce the role information in the model to adapt the property of short text in Chinese sentence. In addition, adding pre-training word emebdding to the model can deal with unknown Chinese words more effectively, and this could reduce the possibility of misidentification. In the end, this thesis compares many kinds of deep learning models and introduces role information to identify dialog act, which is nearly 1.2% higher than the typical neural network model in the telecome domain dialogue dataset.

關鍵字(中)

★ 對話行為
★ 詞向量
★ 深度學習
★ 卷積類神經網路
★ 長短期記憶模型
★ 注意力機制

關鍵字(英)

★ Dialog act
★ Word embedding
★ Deep learning
★ Convolutional neural network
★ Long-Short Term Memory
★ Attention mechanism

論文目次

摘要 I
ABSTRACT II
致謝 III
目錄 IV
圖目錄 VI
表目錄 VII
第一章緒論 1
1.1研究背景 1
1.2研究動機 2
1.3章節概要 3
第二章文獻探討 4
2.1 對話行為分類 4
2.1.1對話系統 4
2.1.2自然語言理解 5
2.1.3對話行為 6
2.2 深度學習 6
2.2.1 詞向量(Word2vec) 6
2.2.2 卷積神經網路(Convolutional Neural Network) 7
2.2.3 循環神經網路(Recurrent Neural Network) 7
2.2.4 GRU(Gated Recurrent Unit) 8
2.2.5 長短期記憶網路(Long short-term memory) 9
2.2.6 注意力機制(Attention mechanism) 11
第三章系統架構 12
3.1模組架構 12
3.1.1前處理模組 12
3.1.2預訓練詞向量模組 13
3.1.3 DNN模型框架 13
3.2模型描述 14
3.2.1句子編碼器(Sentence Encoder) 14
3.2.2前後文編碼器(Context Encoder) 15
3.2.3分類器(Classifier) 15
3.2.4模型一句子模型 15
3.2.5模型二句子結合角色資訊模型 16
3.2.6模型三前後文模型 17
3.2.7模型四前後文結合角色資訊模型 17
第四章實驗方法 19
4.1資料描述 19
4.1.1 對話資料 19
4.1.2 預訓練詞向量 23
4.2參數說明 23
4.3實驗結果 24
4.3.1 Baseline 24
4.3.2 句子模型 – 模型一、模型二 25
4.3.3 前後文模型 – 模型三、模型四 26
4.4錯誤分析 27
第五章結論與未來研究 29
5.1實驗成果 29
5.2未來方向 29
附錄 30
參考文獻 36

參考文獻

Young, S. (1999). Probabilistic methods in spoken dialogue systems. Paper presented at the Philosophical Transactions of the Royal Society (Series A.
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016). Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models. Paper presented at the AAAI.
Wen, T.-H., Vandyke, D., Mrksic, N., Gasic, M., Rojas-Barahona, L. M., Su, P.-H., . . . Young, S. (2016). A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562.
Fernandez, R., & Picard, R. W. (2002). Dialog act classification from prosodic features using support vector machines. Paper presented at the Speech Prosody 2002, International Conference.
Barahona, L. M. R., Gasic, M., Mrkšić, N., Su, P.-H., Ultes, S., Wen, T.-H., & Young, S. (2016). Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding. arXiv preprint arXiv:1610.04120.
Lee, J. Y., & Dernoncourt, F. (2016). Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:1603.03827.
Traum, D. R. (1999). Speech acts for dialogue agents Foundations of rational agency (pp. 169-201): Springer.
Meng, L., & Huang, M. (2018). Dialogue Intent Classification with Long Short-Term Memory Networks, Cham.
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992). SWITCHBOARD: Telephone speech corpus for research and development. Paper presented at the Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on.
Shriberg, E., Dhillon, R., Bhagat, S., Ang, J., & Carvey, H. (2004). The ICSI meeting recorder dialog act (MRDA) corpus. Retrieved from
Joty, S., Carenini, G., & Lin, C.-Y. (2011). Unsupervised modeling of dialog acts in asynchronous conversations. Paper presented at the IJCAI Proceedings-International Joint Conference on Artificial Intelligence.
Kim, S. N., Cavedon, L., & Baldwin, T. (2010). Classifying dialogue acts in one-on-one live chats. Paper presented at the Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing.
Kim, S. N., Cavedon, L., & Baldwin, T. (2012). Classifying Dialogue Acts in Multi-party Live Chats. Paper presented at the PACLIC.
Shen, S.-s., & Lee, H.-y. (2016). Neural attention models for sequence classification: Analysis and application to key term extraction and dialogue act detection. arXiv preprint arXiv:1604.00077.
Ortega, D., & Vu, N. T. (2017). Neural-based Context Representation Learning for Dialog Act Classification. arXiv preprint arXiv:1708.02561.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Paper presented at the Advances in neural information processing systems.
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4), 541-551.
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug), 2493-2537.
Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.
Zhang, Y., & Wallace, B. (2015). A sensitivity analysis of (and practitioners′ guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985). Learning internal representations by error propagation. Retrieved from
Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179-211.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016). Hierarchical Attention Networks for Document Classification. Paper presented at the HLT-NAACL.
Kowsari, K., Brown, D. E., Heidarysafa, M., Meimandi, K. J., Gerber, M. S., & Barnes, L. E. (2017). Hdltex: Hierarchical deep learning for text classification. arXiv preprint arXiv:1709.08267.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
Yin, W., Kann, K., Yu, M., & Schütze, H. (2017). Comparative Study of CNN and RNN for Natural Language Processing. arXiv preprint arXiv:1702.01923.
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Shi, H., Ushio, T., Endo, M., Yamagami, K., & Horii, N. (2016). A multichannel convolutional neural network for cross-language dialog state tracking. Paper presented at the Spoken Language Technology Workshop (SLT), 2016 IEEE.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
Xiao, Y., & Cho, K. (2016). Efficient character-level document classification by combining convolution and recurrent layers. arXiv preprint arXiv:1602.00367.
Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. Paper presented at the Proceedings of the 31st International Conference on Machine Learning (ICML-14).
Weston, J., Chopra, S., & Bordes, A. (2014). Memory networks. arXiv preprint arXiv:1410.3916.

指導教授

蔡宗翰(Tzong-Han Tsai)

審核日期

2018-1-26

推文