# A Shared Geometry of Difficulty in Multilingual Language Models

arXiv: https://arxiv.org/abs/2601.12731 · HTML: https://mlsanity.com/paper/2601.12731/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability.
- Builds on 6 of its 16 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Placed in

- Mechanistic interpretability / [Knowledge and representations inside LLMs](https://mlsanity.com/mechinterp/knowledge-and-representations-inside-llms/)

## Builds on (most cited first, 6 of 6)

- [The Geometry of Multilingual Language Model Representations](https://mlsanity.com/paper/2205.10964/) (2022)
- [Do Multilingual LLMs Think In English?](https://mlsanity.com/paper/2502.15603/) (2025)
- [How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study](https://mlsanity.com/paper/2402.16061/) (2024)
- [Understanding Cross-Lingual Alignment -- A Survey](https://mlsanity.com/paper/2404.06228/) (2024)
- [Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysis](https://mlsanity.com/paper/2409.14459/) (2024)
- [LLMs Encode How Difficult Problems Are](https://mlsanity.com/paper/2510.18147/) (2025)

## Built on by (most cited first, 2 of 2)

- [More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs](https://mlsanity.com/paper/2608.30463/) (2026)
- [Cross-lingual Calibration of Pre-Generation Success Probes for Multilingual LLM Routing](https://mlsanity.com/paper/2610.06216/) (2026)

## Abstract (arXiv)

Predicting problem-difficulty in large language models (LLMs) refers to estimating how difficult a task is according to the model itself, typically by training linear probes on its internal representations. In this work, we study the multilingual geometry of problem-difficulty in LLMs by training linear probes using the AMC subset of the Easy2Hard benchmark, translated into 21 languages. We found that difficulty-related signals emerge at two distinct stages of the model internals, corresponding to shallow (early-layers) and deep (later-layers) internal representations, that exhibit functionally different behaviors. Probes trained on deep representations achieve high accuracy when evaluated on the same language but exhibit poor cross-lingual generalization. In contrast, probes trained on shallow representations generalize substantially better across languages, despite achieving lower within-language performance. Together, these results suggest that LLMs first form a language-agnostic representation of problem difficulty, which subsequently becomes language-specific. This closely aligns with existing findings in LLM interpretability showing that models tend to operate in an abstract conceptual space before producing language-specific outputs. We demonstrate that this two-stage representational process extends beyond semantic content to high-level meta-cognitive properties such as problem-difficulty estimation.