# No Text Needed: Forecasting MT Quality and Inequity from Fertility and Metadata

arXiv: https://arxiv.org/abs/2509.05425 · HTML: https://mlsanity.com/paper/2509.05425/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 12 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 3 of 3)

- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects](https://mlsanity.com/paper/2309.07445/) (2023)
- [mSTEB: Massively Multilingual Evaluation of LLMs on Speech and Text Tasks](https://mlsanity.com/paper/2506.08400/) (2025)

## Abstract (arXiv)

We show that translation quality can be predicted with surprising accuracy textitwithout ever running the translation system itself. Using only a handful of features, token fertility ratios, token counts, and basic linguistic metadata (language family, script, and region), we can forecast ChrF scores for GPT-4o translations across 203 languages in the FLORES-200 benchmark. Gradient boosting models achieve favorable performance (R2=0.66 for XXrightarrowEnglish and R2=0.72 for EnglishrightarrowXX). Feature importance analyses reveal that typological factors dominate predictions into English, while fertility plays a larger role for translations into diverse target languages. These findings suggest that translation quality is shaped by both token-level fertility and broader linguistic typology, offering new insights for multilingual evaluation and quality estimation.