mlsanityData to 14 Oct 2026

Cross-lingual Calibration of Pre-Generation Success Probes for Multilingual LLM Routing

arXiv 2610.06216 ↗First posted 5 Oct 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Pre-generation success probes estimate response correctness from a language model's hidden activations before decoding, enabling cost-aware routing. While prior work has demonstrated their utility primarily on English inputs, we study their reliability across languages along three dimensions: (1) whether they preserve the ranking of likely successes and failures (DISCRIMINATION); (2) whether they retain probabilities that match observed success frequencies (CALIBRATION); and (3) whether they produce scores comparable enough across candidate models for cost-aware multilingual routing (UTILITY). Using 3,000 MATH problems in 10 languages and 8 open-weight model configurations, we compare cross-lingual transfer from English-trained probes and equal-budget pooled multilingual probes. English-trained probes retain useful cross-lingual discrimination but become less well calibrated after transfer. Pooled multilingual supervision improves both properties and yields more reliable estimates of success. In routing experiments, the pooled router achieves a 0.7% higher test success rate while reducing modeled cost by 13.0% relative to always selecting the model with the highest average success. These results show that multilingual routing requires success estimates that remain well calibrated and comparable across languages and models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 43 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

31 of 43 references.

References · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  7. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  8. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  9. 2019How multilingual is Multilingual BERT?cited by 645
  10. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  11. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  12. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
19 more
  1. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  2. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  3. 2025Universal Model Routing for Efficient LLM Inferencecited by 86
  4. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 85
  5. 2024A Unified Approach to Routing and Cascading for LLMscited by 80
  6. 2025Do Multilingual LLMs Think In English?cited by 78
  7. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 43
  8. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  9. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 37
  10. 2024mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models?cited by 21
  11. 2026LLMs Encode Their Failures: Predicting Success from Pre-Generation Activationscited by 14
  12. 2024Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysiscited by 12
  13. 2025Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Modelscited by 10
  14. 2025LLMs Encode How Difficult Problems Arecited by 7
  15. 2026LLM Router: Rethinking Routing with Prefill Activationscited by 5
  16. 2026Do LLM hallucination detectors suffer from low-resource effect?cited by 4
  17. 2026IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluationcited by 3
  18. 2026A Shared Geometry of Difficulty in Multilingual Language Modelscited by 2
  19. 2026LLMs Encode Relevance as a Layer-Wise Cross-Lingual Signalcited by 1

Built on by

Not cited yet.