mlsanityData to 14 Oct 2026

MTQ-Eval: Multilingual Text Quality Evaluation for Language Models

arXiv 2511.09374 ↗First posted 12 Nov 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The use of large language models (LLMs) for evaluating outputs is becoming an increasingly effective and scalable approach. However, it remains uncertain whether this capability extends beyond task-specific evaluations to more general assessments of text quality, particularly in multilingual contexts. In this study, we introduce, MTQ-Eval, a novel framework for multilingual text quality evaluation that learns from examples of both high- and low-quality texts, adjusting its internal representations. To develop MTQ-Eval, we first automatically generate text quality preference data and then use it to train open-source base LLMs to align with ratings of high- and low-quality text. Our comprehensive evaluation across 115 languages demonstrates the improved performance of the proposed model. Upon further analysis, we find that this enhanced evaluation capability also leads to notable improvements in downstream tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 12 Nov 2025.

Builds on

22 of 38 references.

References · 22

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  4. 2018Neural Network Acceptability Judgmentscited by 958
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  6. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  7. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 347
  8. 2023Using an LLM to Help With Code Understandingcited by 228
  9. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
  10. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
  11. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 88
  12. 2024Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generationcited by 63
10 more
  1. 2024Are LLM-based Evaluators Confusing NLG Quality Criteria?cited by 36
  2. 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
  3. 2024Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignmentcited by 26
  4. 2024What Drives Performance in Multilingual Language Models?cited by 15
  5. 2023MELA: Multilingual Evaluation of Linguistic Acceptabilitycited by 14
  6. 2024How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs?cited by 13
  7. 2023Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmarkcited by 10
  8. 2024RepEval: Effective Text Evaluation with LLM Representationcited by 6
  9. 2024Beyond Data Quantity: Key Factors Driving Performance in Multilingual Language Modelscited by 6
  10. 2025CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferencescited by 3

Built on by

Not cited yet.