# An Effective Approach to Unsupervised Machine Translation

arXiv: https://arxiv.org/abs/1902.01313 · HTML: https://mlsanity.com/paper/1902.01313/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2019.

## Built on by (most cited first, 30 of 37)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [SLaDe: A Portable Small Language Model Decompiler for Optimized Assembly](https://mlsanity.com/paper/2305.12520/) (2023)
- [Multilingual Unsupervised Neural Machine Translation with Denoising Adapters](https://mlsanity.com/paper/2110.10472/) (2021)
- [Modeling Dual Read/Write Paths for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09163/) (2022)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2202.10062/) (2022)
- [Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2103.10531/) (2021)
- [Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation](https://mlsanity.com/paper/2303.15265/) (2023)
- [Unsupervised Melody-to-Lyric Generation](https://mlsanity.com/paper/2305.19228/) (2023)
- [Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport](https://mlsanity.com/paper/2210.14378/) (2022)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- [Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2203.08394/) (2022)
- [Translating away Translationese without Parallel Data](https://mlsanity.com/paper/2310.18830/) (2023)
- [On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation](https://mlsanity.com/paper/2106.15818/) (2021)
- [Integrating Unsupervised Data Generation into Self-Supervised Neural Machine Translation for Low-Resource Languages](https://mlsanity.com/paper/2107.08772/) (2021)
- ["A Passage to India": Pre-trained Word Embeddings for Indian Languages](https://mlsanity.com/paper/2112.13800/) (2021)
- [Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages](https://mlsanity.com/paper/2103.13272/) (2021)
- [Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation](https://mlsanity.com/paper/2209.02821/) (2022)
- [Align before Search: Aligning Ads Image to Text for Accurate Cross-Modal Sponsored Search](https://mlsanity.com/paper/2309.16141/) (2023)
- [An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces](https://mlsanity.com/paper/2109.12640/) (2021)
- [Exploiting Social Media Content for Self-Supervised Style Transfer](https://mlsanity.com/paper/2205.08814/) (2022)
- [A Semi-supervised Approach for a Better Translation of Sentiment in Dialectical Arabic UGT](https://mlsanity.com/paper/2210.11899/) (2022)
- [Text Attribute Control via Closed-Loop Disentanglement](https://mlsanity.com/paper/2312.00277/) (2023)
- [Towards Unsupervised Speech Recognition at the Syllable-Level](https://mlsanity.com/paper/2510.03639/) (2025)
- [Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings](https://mlsanity.com/paper/2103.06689/) (2021)
- [Domain Mismatch Doesn't Always Prevent Cross-Lingual Transfer Learning](https://mlsanity.com/paper/2211.16671/) (2022)
- [Effective Self-Mining of In-Context Examples for Unsupervised Machine Translation with LLMs](https://mlsanity.com/paper/2410.11006/) (2024)
- [Embedding-Enhanced Giza++: Improving Alignment in Low- and High- Resource Scenarios Using Embedding Space Geometry](https://mlsanity.com/paper/2104.08721/) (2021)
- [Icelandic Parallel Abstracts Corpus](https://mlsanity.com/paper/2108.05289/) (2021)
- [Paraphrase Generation as Unsupervised Machine Translation](https://mlsanity.com/paper/2109.02950/) (2021)

## Abstract (arXiv)

While machine translation has traditionally relied on large amounts of parallel corpora, a recent research line has managed to train both Neural Machine Translation (NMT) and Statistical Machine Translation (SMT) systems using monolingual corpora only. In this paper, we identify and address several deficiencies of existing unsupervised SMT approaches by exploiting subword information, developing a theoretically well founded unsupervised tuning method, and incorporating a joint refinement procedure. Moreover, we use our improved SMT system to initialize a dual NMT model, which is further fine-tuned through on-the-fly back-translation. Together, we obtain large improvements over the previous state-of-the-art in unsupervised machine translation. For instance, we get 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more than the (supervised) shared task winner back in 2014.