mlsanityData to 14 Oct 2026

XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks

arXiv 2605.30788 ↗First posted 29 May 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a set of synthetic algorithmic tasks to detect cross-lingual gaps in the abilities of large language models. Our benchmark is commensurate across languages, since it requires models to perform the same underlying task in different languages; scalable, since each task can be generated at varying levels of complexity allowing it to be adapted to models with different capabilities; quantifiable, since every task admits an objective notion of correctness; and transparent, since tasks are generated from simple templates that can be readily audited for translation errors. Because our benchmark focuses on algorithmic tasks, differential performance is a sufficient -- but not necessary -- indicator of cross-lingual gaps. Nevertheless, we show through extensive experiments that our benchmark exposes persistent cross-lingual gaps in multiple state-of-the-art models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

25 of 40 references.

References · 25

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  3. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  4. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 191
  5. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  6. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 79
  7. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  8. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languagescited by 55
  9. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 49
  10. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  11. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
  12. 2025A Survey of Multilingual Reasoning in Language Modelscited by 33
13 more
  1. 2025SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulascited by 33
  2. 2025The Bitter Lesson Learned from 2,000+ Multilingual Benchmarkscited by 31
  3. 2025One ruler to measure them all: Benchmarking multilingual long-context language modelscited by 19
  4. 2025XIFBench: Evaluating Large Language Models on Multilingual Instruction Followingcited by 11
  5. 2025Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Resultscited by 10
  6. 2025SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoningcited by 8
  7. 2024Training and Evaluating Language Models with Template-based Data Generationcited by 6
  8. 2024Foundation Models for Low-Resource Language Education (Vision Paper)cited by 5
  9. 2025Measuring AI Diffusion: A Population-Normalized Metric for Tracking Global AI Usagecited by 5
  10. 2026ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Modelscited by 5
  11. 2026Multilingual Reasoning Gym: Multilingual Scaling of Procedural Reasoning Environmentscited by 5
  12. 2025MultiZebraLogic: A Multilingual Logical Reasoning Benchmarkcited by 1
  13. 2026A model of errors in transformerscited by 1

Built on by

Not cited yet.