# XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks

arXiv: https://arxiv.org/abs/2605.30788 · HTML: https://mlsanity.com/paper/2605.30788/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 29 May 2026.

## Builds on (most cited first, 25 of 25)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity](https://mlsanity.com/paper/2506.06941/) (2025)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://mlsanity.com/paper/2412.03304/) (2024)
- [REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2505.24760/) (2025)
- [MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation](https://mlsanity.com/paper/2503.10497/) (2025)
- [MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks](https://mlsanity.com/paper/2311.07463/) (2023)
- [All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages](https://mlsanity.com/paper/2411.16508/) (2024)
- [GraphArena: Evaluating and Exploring Large Language Models on Graph Computation](https://mlsanity.com/paper/2407.00379/) (2024)
- [Crosslingual Reasoning through Test-Time Scaling](https://mlsanity.com/paper/2505.05408/) (2025)
- [Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts](https://mlsanity.com/paper/2306.11372/) (2023)
- [A Survey of Multilingual Reasoning in Language Models](https://mlsanity.com/paper/2502.09457/) (2025)
- [SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulas](https://mlsanity.com/paper/2505.14615/) (2025)
- [The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks](https://mlsanity.com/paper/2504.15521/) (2025)
- [One ruler to measure them all: Benchmarking multilingual long-context language models](https://mlsanity.com/paper/2503.01996/) (2025)
- [XIFBench: Evaluating Large Language Models on Multilingual Instruction Following](https://mlsanity.com/paper/2503.07539/) (2025)
- [Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Results](https://mlsanity.com/paper/2511.05162/) (2025)
- [SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning](https://mlsanity.com/paper/2505.16368/) (2025)
- [Training and Evaluating Language Models with Template-based Data Generation](https://mlsanity.com/paper/2411.18104/) (2024)
- [Foundation Models for Low-Resource Language Education (Vision Paper)](https://mlsanity.com/paper/2412.04774/) (2024)
- [Measuring AI Diffusion: A Population-Normalized Metric for Tracking Global AI Usage](https://mlsanity.com/paper/2511.02781/) (2025)
- [ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models](https://mlsanity.com/paper/2602.20117/) (2026)
- [Multilingual Reasoning Gym: Multilingual Scaling of Procedural Reasoning Environments](https://mlsanity.com/paper/2603.10793/) (2026)
- [MultiZebraLogic: A Multilingual Logical Reasoning Benchmark](https://mlsanity.com/paper/2511.03553/) (2025)
- [A model of errors in transformers](https://mlsanity.com/paper/2601.14175/) (2026)

## Abstract (arXiv)

We introduce a set of synthetic algorithmic tasks to detect cross-lingual gaps in the abilities of large language models. Our benchmark is commensurate across languages, since it requires models to perform the same underlying task in different languages; scalable, since each task can be generated at varying levels of complexity allowing it to be adapted to models with different capabilities; quantifiable, since every task admits an objective notion of correctness; and transparent, since tasks are generated from simple templates that can be readily audited for translation errors. Because our benchmark focuses on algorithmic tasks, differential performance is a sufficient -- but not necessary -- indicator of cross-lingual gaps. Nevertheless, we show through extensive experiments that our benchmark exposes persistent cross-lingual gaps in multiple state-of-the-art models.