# Unsupervised Statistical Machine Translation

arXiv: https://arxiv.org/abs/1809.01272 · HTML: https://mlsanity.com/paper/1809.01272/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2018.

## Built on by (most cited first, 30 of 51)

- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Break-It-Fix-It: Unsupervised Learning for Program Repair](https://mlsanity.com/paper/2106.06600/) (2021)
- [Code Translation with Compiler Representations](https://mlsanity.com/paper/2207.03578/) (2022)
- [Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review](https://mlsanity.com/paper/2307.02503/) (2023)
- [Inducing Positive Perspectives with Text Reframing](https://mlsanity.com/paper/2204.02952/) (2022)
- [Encoder-Decoder Diffusion Language Models for Efficient Training and Inference](https://mlsanity.com/paper/2510.22852/) (2025)
- [RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation](https://mlsanity.com/paper/2412.17744/) (2024)
- [Semi-Supervised Graph-to-Graph Translation](https://mlsanity.com/paper/2103.08827/) (2021)
- [Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2103.10531/) (2021)
- [An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languages](https://mlsanity.com/paper/2109.00165/) (2021)
- [Automatic Intent-Slot Induction for Dialogue Systems](https://mlsanity.com/paper/2103.08886/) (2021)
- [Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2205.11116/) (2022)
- [Graphusion: A RAG Framework for Knowledge Graph Construction with a Global Perspective](https://mlsanity.com/paper/2410.17600/) (2024)
- [Unsupervised Cross-Domain Prerequisite Chain Learning using Variational Graph Autoencoders](https://mlsanity.com/paper/2105.03505/) (2021)
- [Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval](https://mlsanity.com/paper/2104.08801/) (2021)
- [Extremely low-resource machine translation for closely related languages](https://mlsanity.com/paper/2105.13065/) (2021)
- [MulZDG: Multilingual Code-Switching Framework for Zero-shot Dialogue Generation](https://mlsanity.com/paper/2208.08629/) (2022)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- ["Wikily" Supervised Neural Translation Tailored to Cross-Lingual Tasks](https://mlsanity.com/paper/2104.08384/) (2021)
- [Translating away Translationese without Parallel Data](https://mlsanity.com/paper/2310.18830/) (2023)
- [Cross-lingual hate speech detection based on multilingual domain-specific word embeddings](https://mlsanity.com/paper/2104.14728/) (2021)
- ["A Passage to India": Pre-trained Word Embeddings for Indian Languages](https://mlsanity.com/paper/2112.13800/) (2021)
- [Sentence Alignment with Parallel Documents Facilitates Biomedical Machine Translation](https://mlsanity.com/paper/2104.08588/) (2021)
- [Crosslingual Embeddings are Essential in UNMT for Distant Languages: An English to IndoAryan Case Study](https://mlsanity.com/paper/2106.04995/) (2021)
- [HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints](https://mlsanity.com/paper/2109.04443/) (2021)
- [Efficient Variational Graph Autoencoders for Unsupervised Cross-domain Prerequisite Chains](https://mlsanity.com/paper/2109.08722/) (2021)
- [Dim Wihl Gat Tun: The Case for Linguistic Expertise in NLP for Underdocumented Languages](https://mlsanity.com/paper/2203.09632/) (2022)
- [Computational Sarcasm Analysis on Social Media: A Systematic Review](https://mlsanity.com/paper/2209.06170/) (2022)
- [Text Attribute Control via Closed-Loop Disentanglement](https://mlsanity.com/paper/2312.00277/) (2023)

## Abstract (arXiv)

While modern machine translation has relied on large parallel corpora, a recent line of work has managed to train Neural Machine Translation (NMT) systems from monolingual corpora only (Artetxe et al., 2018c; Lample et al., 2018). Despite the potential of this approach for low-resource settings, existing systems are far behind their supervised counterparts, limiting their practical interest. In this paper, we propose an alternative approach based on phrase-based Statistical Machine Translation (SMT) that significantly closes the gap with supervised systems. Our method profits from the modular architecture of SMT: we first induce a phrase table from monolingual corpora through cross-lingual embedding mappings, combine it with an n-gram language model, and fine-tune hyperparameters through an unsupervised MERT variant. In addition, iterative backtranslation improves results further, yielding, for instance, 14.08 and 26.22 BLEU points in WMT 2014 English-German and English-French, respectively, an improvement of more than 7-10 BLEU points over previous unsupervised systems, and closing the gap with supervised SMT (Moses trained on Europarl) down to 2-5 BLEU points. Our implementation is available at https://github.com/artetxem/monoses