Conditional Contrastive Learning for Multilingual Neural Machine Translation

NCUIR > School of Management at National Central University > Graduate Institute of Information Management > Electronic Thesis & Dissertation > Item 987654321/89905

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/89905

Title:	Conditional Contrastive Learning for Multilingual Neural Machine Translation
Authors:	邱睿揚;Qiu, Rui-Yang
Contributors:	資訊管理學系
Keywords:	機器翻譯;深度神經網絡;對比式學習;負採樣;machine translation;deep neural network;contrastive learning;negative sampling
Date:	2022-08-26
Issue Date:	2022-10-04 12:04:11 (UTC+8)
Publisher:	國立中央大學
Abstract:	翻譯任務一直是自然語言領域備受關注的話題，即使之前知名翻譯任務的許多研究都對翻譯性能和效果做出了貢獻，但它們僅限於處理單一對或“以英語為中心”。最近，越來越多的研究針對多語言、非以英語為中心，希望構建多語言機器翻譯系統。現行多語言機器翻譯方法仍然在訓練中存在資料不均衡問題。近期有研究使用對比式學習方法來縮小語言之間的表示差距，藉此提升多語言機器翻譯性能。另一方面有研究於電腦視覺領域對當前知名的對比式學習框架負採樣的方法提出存疑，認為沒有條件選取負樣本會導致學習不穩定，因此假設給予條件選取負樣本能協助提升對比式學習之性能，並且證實其有效性。由於目前沒有任務在多語言翻譯任務上探討對比式學習的負採樣問題，因此本研究想探討於對比式學習中設定條件選取負樣本的方法。從實驗結果中，我們發現我們提出的條件式對比學習方法在監督定向翻譯結果及 zero-shot 翻譯結果不如我們預期。當前的數據量不足使屬於自監督式學習的對比式學習方法無法有效改進多語言機器翻譯，使得多任務學習模型翻譯結果難以超越監督式學習的模型。我們進一步分析了使用我們提出的模型學習出來的共享句子表示式，我們也將此共用表達式與單一任務學習的 m-Transformer 和對比式學習模型做視覺化比較，並且證明我們的共用表示式是能夠有效學習跨語言的共享句子表示式，我們取得與對比式學習模型類似的結果，我們認為我們的優勢在於在於在學習目標事先經過採樣而不像先前作法提取全部對象進行學習，使用較少學習對象能夠降低學習不穩定問題。 ;The translation task has always been a topic of great concern in the field of natural language, and even though many previous studies of well-known translation tasks have contributed to translation performance and effectiveness, they have been limited to dealing with single pair or "English-centric" tasks. Recently, more and more research has been conducted on multilingual, non-English-centric, machine translation systems. Current multilingual machine translation methods still suffer from data imbalance in training. Recent studies have used contrastive learning methods to reduce the representation gap between different languages to improve the performance of multilingual machine translation. On the other hand, a study in the field of computer vision has questioned the well- known contrastive learning framework of negative sampling, arguing that unconditional selection of negative samples can cause unstable learning, and therefore hypothesizing that conditional selection of negative samples can help improve the performance of contrastive learning and prove its effectiveness. Since there is no task to discuss the problem of negative sampling in contrastive learning on multilingual translation tasks, this study aims to discuss the method of setting conditional negative samples in contrastive learning. From the experimental results, we found that our proposed conditional contrastive learning method was not as effective as we expected in supervised directional translation. Insufficient amount of training data makes the contrastive learning method of self-supervised learning ineffective in improving multilingual machine translation, making it difficult for multi-task learning models to outperform supervised learning models in translation. We further analyze the shared sentence representations learned using our proposed model. We also visualize and compare these shared representations with m-Transformer and contrastive learning models for single-task learning, and demonstrate that our shared representations can be effective in learning shared sentence representations across languages. We obtained similar results to the contrastive learning model, and we believe that our advantage lies in the fact that the learning targets are sampled beforehand instead of extracting all objects for learning as in the previous approach, and that using fewer learning objects reduces the learning instability problem.
Appears in Collections:	[Graduate Institute of Information Management] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	65	View/Open

社群 sharing

Loading...