mlsanityData to 7 Oct 2026

Cross-Lingual LLM-Judge Transfer via Evaluation Decomposition

arXiv 2603.18557 ↗First posted 19 Mar 2026cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

As large language models are increasingly deployed across diverse real-world applications, extending automated evaluation beyond English has become a critical challenge. Existing evaluation approaches are predominantly English-focused, and adapting them to other languages is hindered by the scarcity and cost of human-annotated judgments in most languages. We introduce a decomposition-based evaluation framework built around a Universal Criteria Set (UCS). UCS consists of a shared, language-agnostic set of evaluation dimensions, producing an interpretable intermediate representation that supports cross-lingual transfer with minimal supervision. Experiments on multiple faithfulness tasks across languages and model backbones demonstrate consistent improvements over strong baselines without requiring target-language annotations.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

28 of 42 references.

References · 28

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  3. 2020Concept Bottleneck Modelscited by 1,022
  4. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  5. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  6. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  7. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  8. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  9. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
  10. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
  11. 2024CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklistscited by 87
  12. 2024FineSurE: Fine-grained Summarization Evaluation using LLMscited by 83
16 more
  1. 2018BLEU is Not Suitable for the Evaluation of Text Simplificationcited by 78
  2. 2025Do Multilingual LLMs Think In English?cited by 75
  3. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  4. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  5. 2024Concept Bottleneck Large Language Modelscited by 41
  6. 2025RocketEval: Efficient Automated LLM Evaluation via Grading Checklistcited by 37
  7. 2025M-Prometheus: A Suite of Open Multilingual LLM Judgescited by 37
  8. 2023Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experimentcited by 35
  9. 2023Auxiliary Losses for Learning Generalizable Concept-based Modelscited by 35
  10. 2025How Reliable is Multilingual LLM-as-a-Judge?cited by 29
  11. 2024Self-rationalization improves LLM as a fine-grained judgecited by 25
  12. 2024Cross-Lingual Auto Evaluation for Assessing Multilingual LLMscited by 17
  13. 2024DEBATE: Devil's Advocate-Based Assessment and Text Evaluationcited by 16
  14. 2024M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluationcited by 13
  15. 2025Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluatorscited by 5
  16. 2025Multilingual Self-Taught Faithfulness Evaluatorscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluationcited by 1
  2. 2026A Survey on Rubric-Guided Reinforcement Learning for Language Modelscited by 1
  3. 2026Rank Reversal in Multilingual LLM Judges: A Label-Free Double-Centering Calibratorcited by 0