# A Theory of Unsupervised Translation Motivated by Understanding Animal Communication

arXiv: https://arxiv.org/abs/2211.11081 · HTML: https://mlsanity.com/paper/2211.11081/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 20 Nov 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Leveraging Automated Unit Tests for Unsupervised Code Translation](https://mlsanity.com/paper/2110.06773/) (2021)
- [Facebook AI WMT21 News Translation Task Submission](https://mlsanity.com/paper/2108.03265/) (2021)
- [When Does Unsupervised Machine Translation Work?](https://mlsanity.com/paper/2004.05516/) (2020)
- [Language Model Prior for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2004.14928/) (2020)
- [When and Why is Unsupervised Neural Machine Translation Useless?](https://mlsanity.com/paper/2004.10581/) (2020)
- [Probabilistic Tools for the Analysis of Randomized Optimization Heuristics](https://mlsanity.com/paper/1801.06733/) (2018)
- [Unsupervised Neural Machine Translation with Generative Language Models Only](https://mlsanity.com/paper/2110.05448/) (2021)
- [Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders](https://mlsanity.com/paper/1608.02996/) (2016)
- [Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2203.08394/) (2022)
- [Graph matching beyond perfectly-overlapping ErdHos--R'enyi random graphs](https://mlsanity.com/paper/2006.03284/) (2020)

## Built on by (most cited first, 6 of 6)

- [Unsupervised Translation of Emergent Communication](https://mlsanity.com/paper/2502.07552/) (2025)
- [LLMs can hide text in other text of the same length](https://mlsanity.com/paper/2510.20075/) (2025)
- [WhAM: Towards A Translative Model of Sperm Whale Vocalization](https://mlsanity.com/paper/2512.02206/) (2025)
- [UniGlyph: A Seven-Segment Script for Universal Language Representation](https://mlsanity.com/paper/2410.08974/) (2024)
- [On Non-interactive Evaluation of Animal Communication Translators](https://mlsanity.com/paper/2510.15768/) (2025)
- [Meta-ethics and AI: exploring the novel meta-ethical questions in the era of AI](https://mlsanity.com/paper/2609.01685/) (2026)

## Abstract (arXiv)

Neural networks are capable of translating between languages -- in some cases even between two languages where there is little or no access to parallel translations, in what is known as Unsupervised Machine Translation (UMT). Given this progress, it is intriguing to ask whether machine learning tools can ultimately enable understanding animal communication, particularly that of highly intelligent animals. We propose a theoretical framework for analyzing UMT when no parallel translations are available and when it cannot be assumed that the source and target corpora address related subject domains or posses similar linguistic structure. We exemplify this theory with two stylized models of language, for which our framework provides bounds on necessary sample complexity; the bounds are formally proven and experimentally verified on synthetic data. These bounds show that the error rates are inversely related to the language complexity and amount of common ground. This suggests that unsupervised translation of animal communication may be feasible if the communication system is sufficiently complex.