# Lost in Translation: Large Language Models in Non-English Content Analysis

arXiv: https://arxiv.org/abs/2306.07377 · HTML: https://mlsanity.com/paper/2306.07377/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 149 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 30 of 60)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Predictability and Surprise in Large Generative Models](https://mlsanity.com/paper/2202.07785/) (2022)
- [Are All Languages Created Equal in Multilingual BERT?](https://mlsanity.com/paper/2005.09093/) (2020)
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Emerging Cross-lingual Structure in Pretrained Language Models](https://mlsanity.com/paper/1911.01464/) (2019)

## Built on by (most cited first, 30 of 35)

- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [Unintended Impacts of LLM Alignment on Global Representation](https://mlsanity.com/paper/2402.15018/) (2024)
- [How Does Quantization Affect Multilingual LLMs?](https://mlsanity.com/paper/2407.03211/) (2024)
- [Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning](https://mlsanity.com/paper/2507.05418/) (2025)
- [Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs](https://mlsanity.com/paper/2502.09331/) (2025)
- [Shortcomings of LLMs for Low-Resource Translation: Retrieval and Understanding are Both the Problem](https://mlsanity.com/paper/2406.15625/) (2024)
- [The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biases](https://mlsanity.com/paper/2401.10535/) (2024)
- [Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM](https://mlsanity.com/paper/2503.14603/) (2025)
- [Risks and Opportunities of Open-Source Generative AI](https://mlsanity.com/paper/2405.08597/) (2024)
- [Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications?](https://mlsanity.com/paper/2403.04792/) (2024)
- [One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers](https://mlsanity.com/paper/2506.10766/) (2025)
- [The Multilingual Divide and Its Impact on Global AI Safety](https://mlsanity.com/paper/2505.21344/) (2025)
- [LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models](https://mlsanity.com/paper/2505.19240/) (2025)
- [Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers](https://mlsanity.com/paper/2502.17882/) (2025)
- [Mechanistic Interpretability for Large Language Model Alignment: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2602.11180/) (2026)
- [Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventories](https://mlsanity.com/paper/2501.06143/) (2025)
- [Decoding the Diversity: A Review of the Indic AI Research Landscape](https://mlsanity.com/paper/2406.09559/) (2024)
- [The Script Tax: Measuring Tokenization-Driven Efficiency and Latency Disparities in Multilingual Language Models](https://mlsanity.com/paper/2602.11174/) (2026)
- [How Good is Your Wikipedia? Auditing Data Quality for Low-resource and Multilingual NLP](https://mlsanity.com/paper/2411.05527/) (2024)
- [Text2Cypher Across Languages: Evaluating and Finetuning LLMs](https://mlsanity.com/paper/2506.21445/) (2025)
- [How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting](https://mlsanity.com/paper/2507.22923/) (2025)
- [A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem](https://mlsanity.com/paper/2505.08148/) (2025)
- [What if I ask in textitalia lingua? Measuring Functional Similarity Across Languages](https://mlsanity.com/paper/2509.04032/) (2025)
- [Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora](https://mlsanity.com/paper/2605.22660/) (2026)
- [ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality](https://mlsanity.com/paper/2402.18797/) (2024)
- [Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models](https://mlsanity.com/paper/2501.15571/) (2025)
- [Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Models](https://mlsanity.com/paper/2502.00826/) (2025)
- [Multi-granular Training Strategies for Robust Multi-hop Reasoning Over Noisy and Heterogeneous Knowledge Sources](https://mlsanity.com/paper/2502.05944/) (2025)
- [KurdSTS: The Kurdish Semantic Textual Similarity](https://mlsanity.com/paper/2510.02336/) (2025)

## Abstract (arXiv)

In recent years, large language models (e.g., Open AI's GPT-4, Meta's LLaMa, Google's PaLM) have become the dominant approach for building AI systems to analyze and generate language online. However, the automated systems that increasingly mediate our interactions online -- such as chatbots, content moderation systems, and search engines -- are primarily designed for and work far more effectively in English than in the world's other 7,000 languages. Recently, researchers and technology companies have attempted to extend the capabilities of large language models into languages other than English by building what are called multilingual language models. In this paper, we explain how these multilingual language models work and explore their capabilities and limits. Part I provides a simple technical explanation of how large language models work, why there is a gap in available data between English and other languages, and how multilingual language models attempt to bridge that gap. Part II accounts for the challenges of doing content analysis with large language models in general and multilingual language models in particular. Part III offers recommendations for companies, researchers, and policymakers to keep in mind when considering researching, developing and deploying large and multilingual language models.