# Translation in the Wild

arXiv: https://arxiv.org/abs/2505.23548 · HTML: https://mlsanity.com/paper/2505.23548/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 60 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 30 of 38)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Emerging Cross-lingual Structure in Pretrained Language Models](https://mlsanity.com/paper/1911.01464/) (2019)
- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://mlsanity.com/paper/2304.04675/) (2023)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Towards Making the Most of ChatGPT for Machine Translation](https://mlsanity.com/paper/2303.13780/) (2023)
- [The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://mlsanity.com/paper/2308.07286/) (2023)
- [Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models](https://mlsanity.com/paper/2204.08110/) (2022)
- [Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://mlsanity.com/paper/2303.13809/) (2023)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capability](https://mlsanity.com/paper/2305.10266/) (2023)
- [Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages](https://mlsanity.com/paper/2501.06346/) (2025)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [Exploring Human-Like Translation Strategy with Large Language Models](https://mlsanity.com/paper/2305.04118/) (2023)
- [TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement](https://mlsanity.com/paper/2402.16379/) (2024)
- [Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study](https://mlsanity.com/paper/2502.02481/) (2025)
- [A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism](https://mlsanity.com/paper/2401.05749/) (2024)
- [ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback](https://mlsanity.com/paper/2304.02426/) (2023)
- [Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts](https://mlsanity.com/paper/2409.06790/) (2024)
- [English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings](https://mlsanity.com/paper/2211.06127/) (2022)
- [Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations](https://mlsanity.com/paper/2404.07851/) (2024)

## Built on by (most cited first, 2 of 2)

- [Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL](https://mlsanity.com/paper/2604.20835/) (2026)
- [Translation Indeterminacy and the Distributional Fallacy](https://mlsanity.com/paper/2609.07717/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) excel in translation among other things, demonstrating competitive performance for many language pairs in zero- and few-shot settings. But unlike dedicated neural machine translation models, LLMs are not trained on any translation-related objective. What explains their remarkable translation abilities? Are these abilities grounded in "incidental bilingualism" (Briakou et al. 2023) in training data? Does instruction tuning contribute to it? Are LLMs capable of aligning and leveraging semantically identical or similar monolingual contents from different corners of the internet that are unlikely to fit in a single context window? I offer some reflections on this topic, informed by recent studies and growing user experience. My working hypothesis is that LLMs' translation abilities originate in two different types of pre-training data that may be internalized by the models in different ways. I discuss the prospects for testing the "duality" hypothesis empirically and its implications for reconceptualizing translation, human and machine, in the age of deep learning.