基於序列到序列之問題-答案對生成模型

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：21

、訪客IP：18.116.49.6

姓名

湯珮茹(Pei-Ju Tang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於序列到序列之問題-答案對生成模型
(Generating Question-Answer Pairs from Document with Sequence to Sequence Modeling)

相關論文

★ 行程邀約郵件的辨識與不規則時間擷取之研究	★ NCUFree校園無線網路平台設計及應用服務開發
★ 網際網路半結構性資料擷取系統之設計與實作	★ 非簡單瀏覽路徑之探勘與應用
★ 遞增資料關聯式規則探勘之改進	★ 應用卡方獨立性檢定於關連式分類問題
★ 中文資料擷取系統之設計與研究	★ 非數值型資料視覺化與兼具主客觀的分群
★ 關聯性字組在文件摘要上的探討	★ 淨化網頁：網頁區塊化以及資料區域擷取
★ 問題答覆系統使用語句分類排序方式之設計與研究	★ 時序資料庫中緊密頻繁連續事件型樣之有效探勘
★ 星狀座標之軸排列於群聚視覺化之應用	★ 由瀏覽歷程自動產生網頁抓取程式之研究
★ 動態網頁之樣版與資料分析研究	★ 同性質網頁資料整合之自動化研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

近年來有許多回答問題及產生問題的研究應用於閱讀理解中，並獲得相當不錯的成效。回答問題研究的目標是通過閱讀文章來回答問題，而產生問題研究希望根據文章產生多樣性問題且可作為閱讀測驗的考題。然而，產生問題有時會產生超出文章本身的範圍，以致於機器無法產生解答。本論文為了解決上述問題-答案不成對的問題，提出一解決方案。用戶只要輸入文章及問題類型，系統即可輸出具有品質的問題-答案對，其產生的問題和答案必須關注在相同的輸入文章句子上，並具備流暢度、相關性及正確性。
在本論文中，我們採用基於注意力機制的序列到序列模型，並於訊息編碼中加入階層式輸入。為了解決從龐大字典中產生問題及答案而造成無法收斂，所以輸出問題及答案採用動態字典，除包含常用單詞之外，還有隨文章不同而動態改變的單詞，讓訓練能夠收斂並大幅提升模型能力。所訓練的模型除了能產生問題-答案對之外，也能作為回答問題及產生問題，且效能優於檢索式模型。

摘要(英)

In recent years, many studies in question answering and question generation have been applied in reading comprehension with satisfactory performance. Question answering aims to answer questions by reading documents, and question generation attempts to generate diverse questions from given documents and as reading test questions. However, some questions generated in question generation studies are beyond the scope of the given documents themselves, so that the machine cannot find an appropriate answer. In this thesis, we propose an approach to resolve the above question-answer unpaired situations. Given a document and the question type, the system can output a question-answer pair with quality. The question and answer generated must focus on the input document with fluency, relevance, and correctness.
In this thesis, we use the attention-based sequence-to-sequence model and add hierarchical input to the model encoder. In order to solve the problem that the model generating question and answer from huge vocabulary cannot converge, the output question and answer adopts a dynamic vocabulary, which includes not only commonly used words, but also words dynamically changing with the document. Such approach makes the training to converge and improves model capabilities. The trained model can generate question-answer pairs as well as do question answering and question generation. The resulting performance is better than the retrieval-based model.

關鍵字(中)

★ 閱讀理解
★ 序列到序列
★ 問題-答案對
★ 動態字典

關鍵字(英)

★ Reading Comprehension
★ Sequence to Sequence
★ Question-Answer Pair
★ Dynamic Vocabulary

論文目次

摘要 i
Abstract ii
目錄 iii
圖目錄 iv
表目錄 v
1. 緒論 1
2. 相關研究 5
2.1. 自然語言生成(Natural Language Generation, NLG) 5
2.2. 閱讀理解資料集與評估方法 9
3. 系統架構 12
3.1. 自然語言處理(Natural Language Processing, NLP) 12
3.1.1. 資料處理 12
3.1.2. 詞嵌入 13
3.2. seq2seq 15
3.3. Hierarchical seq2seq 17
3.4. 問題類型信心值 17
4. 實驗與系統效能 19
4.1. 資料準備與評估方法 19
4.2. 模型評估 21
4.3. 因子分析與結果討論 27
5. 結論與未來工作 31
參考文獻 32

參考文獻

[1] Kuyten, P., Bickmore, T., Stoyanchev, S., Piwek, P., Prendinger, H., & Ishizuka, M. (2012, September). Fully automated generation of question-answer pairs for scripted virtual instruction. In International Conference on Intelligent Virtual Agents (pp. 1-14). Springer, Berlin, Heidelberg.
[2] Sukhbaatar, S., Weston, J., & Fergus, R. (2015). End-to-end memory networks. In Advances in neural information processing systems (pp. 2440-2448).
[3] Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merrienboer, B., Joulin, A., & Mikolov, T. (2015). Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698.
[4] Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
[5] Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016). MS MARCO: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
[6] Du, X., Shao, J., & Cardie, C. (2017). Learning to Ask: Neural Question Generation for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Vol. 1, pp. 1342-1352).
[7] Wang, W., Yang, N., Wei, F., Chang, B., & Zhou, M. (2017). Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Vol. 1, pp. 189-198).
[8] Tan, C., Wei, F., Yang, N., Du, B., Lv, W., & Zhou, M. (2017). S-net: From answer extraction to answer generation for machine reading comprehension. arXiv preprint arXiv:1706.04815.
[9] Luong, T., Pham, H., & Manning, C. D. (2015). Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 1412-1421).
[10] Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681.
[11] Mikolov, T., Karafiat, M., Burget, L., ?ernocky, J., & Khudanpur, S. (2010). Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.
[12] Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
[13] Karpathy, A. (2015). The unreasonable effectiveness of recurrent neural networks. Andrej Karpathy blog. http://karpathy.github.io/2015/05/21/rnn-effectiveness/
[14] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112).
[15] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
[16] OpenNMT, http://opennmt.net/
[17] TensorFlow, https://www.tensorflow.org/
[18] Wang, T., Yuan, X., & Trischler, A. (2017). A Joint Model for Question Answering and Question Generation. arXiv preprint arXiv:1706.01450.
[19] Indurthi, S., Raghu, D., Khapra, M. M., & Joshi, S. (2017). Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers (Vol. 1, pp. 376-385).
[20] WikiAnswers, http://knowitall.cs.washington.edu/oqa/data/wikianswers/
[21] Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002, July). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics.
[22] Lin, C. Y. (2004). Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out.
[23] Good, I. J. (1953). The population frequencies of species and the estimation of population parameters. Biometrika, 40(3-4), 237-264.
[24] Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.
[25] Pagliardini, M., Gupta, P., & Jaggi, M. (2017). Unsupervised learning of sentence embeddings using compositional n-gram features. arXiv preprint arXiv:1703.02507.
[26] Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
[27] Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5), 378.
[28] Benesty, J., Chen, J., Huang, Y., & Cohen, I. (2009). Pearson correlation coefficient. In Noise reduction in speech processing (pp. 1-4). Springer Berlin Heidelberg.

指導教授

張嘉惠

審核日期

2018-7-20

推文