Data to 7 Oct 2026

On Using Monolingual Corpora in Neural Machine Translation

arXiv 1503.03535 ↗First posted 11 Mar 2015cs.CLcited by 149

Not on the map. Why?

Abstract (arXiv)

Recent work on end-to-end neural network-based architectures for machine translation has shown promising results for En-Fr and En-De translation. Arguably, one of the major factors behind this success has been the availability of high quality parallel corpora. In this work, we investigate how to leverage abundant monolingual corpora for neural machine translation. Compared to a phrase-based and hierarchical baseline, we obtain up to 1.96 BLEU improvement on the low-resource language pair Turkish-English, and 1.59 BLEU on the focused domain task of Chinese-English chat messages. While our method was initially targeted toward such tasks with less parallel data, we show that it also extends to high resource languages such as Cs-En and De-En where we obtain an improvement of 0.39 and 0.47 BLEU scores over the neural machine translation baselines, respectively.

Key facts. Cited by 149 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 39; 2023: 30; 2024: 13; 2025: 5; 2026: 5.

149 papers.

Sort:

By area:

2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
2023End-to-End Speech Recognition: A Surveycited by 127
2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 100
2021Survey of Low-Resource Machine Translationcited by 92
2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
2024Decoding-time Realignment of Language Modelscited by 67
2021Neural Machine Translation with Monolingual Translation Memorycited by 61

38 more

2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
2022SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-trainingcited by 40
2021The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translationcited by 38
2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
2022RescoreBERT: Discriminative Speech Recognition Rescoring with BERTcited by 34
2021Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERTcited by 31
2021On the limit of English conversational speech recognitioncited by 30
2021Librispeech Transducer Model with Internal Language Model Prior Correctioncited by 26
2021Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Modelscited by 25
2021Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognitioncited by 24
2021Injecting Text in Self-Supervised Speech Pretrainingcited by 24
2023An Overview on Language Models: Recent Developments and Outlookcited by 22
2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
2021Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMTcited by 21
2021Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLPcited by 21
2022LAE: Language-Aware Encoder for Monolingual and Multilingual ASRcited by 21
2021Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognitioncited by 19
2021Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognitioncited by 17
2021On Language Model Integration for RNN Transducer based Speech Recognitioncited by 17
2022Modular Hybrid Autoregressive Transducercited by 17
2021Meta Back-translationcited by 16
2021DEEP: DEnoising Entity Pre-training for Neural Machine Translationcited by 15
2022Efficient Training of Neural Transducer for Speech Recognitioncited by 14
2023Large-scale Language Model Rescoring on Long-form Datacited by 14
2021Towards the Unseen: Iterative Text Recognition by Distilling from Errorscited by 12
2022Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Datacited by 12
2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
2022Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languagescited by 12
2022Non-Parametric Domain Adaptation for End-to-End Speech Translationcited by 12
2023Two Stage Contextual Word Filtering for Context bias in Unified Streaming and Non-streaming Transducercited by 12
2023Document-Level Language Models for Machine Translationcited by 12
2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
2023Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillationcited by 11

50 most cited of 149 All 149 (JSON)