# Cross-Lingual LLM-Judge Transfer via Evaluation Decomposition

arXiv: https://arxiv.org/abs/2603.18557 · HTML: https://mlsanity.com/paper/2603.18557/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Builds on (most cited first, 28 of 28)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklists](https://mlsanity.com/paper/2403.18771/) (2024)
- [FineSurE: Fine-grained Summarization Evaluation using LLMs](https://mlsanity.com/paper/2407.00908/) (2024)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [Do Multilingual LLMs Think In English?](https://mlsanity.com/paper/2502.15603/) (2025)
- [A Survey of Large Language Model Agents for Question Answering](https://mlsanity.com/paper/2503.19213/) (2025)
- [Concept Bottleneck Large Language Models](https://mlsanity.com/paper/2412.07992/) (2024)
- [mFACE: Multilingual Summarization with Factual Consistency Evaluation](https://mlsanity.com/paper/2212.10622/) (2022)
- [RocketEval: Efficient Automated LLM Evaluation via Grading Checklist](https://mlsanity.com/paper/2503.05142/) (2025)
- [M-Prometheus: A Suite of Open Multilingual LLM Judges](https://mlsanity.com/paper/2504.04953/) (2025)
- [Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment](https://mlsanity.com/paper/2307.03744/) (2023)
- [Auxiliary Losses for Learning Generalizable Concept-based Models](https://mlsanity.com/paper/2311.11108/) (2023)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [Self-rationalization improves LLM as a fine-grained judge](https://mlsanity.com/paper/2410.05495/) (2024)
- [Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs](https://mlsanity.com/paper/2410.13394/) (2024)
- [DEBATE: Devil's Advocate-Based Assessment and Text Evaluation](https://mlsanity.com/paper/2405.09935/) (2024)
- [M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation](https://mlsanity.com/paper/2412.20127/) (2024)
- [Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluators](https://mlsanity.com/paper/2503.04360/) (2025)
- [Multilingual Self-Taught Faithfulness Evaluators](https://mlsanity.com/paper/2507.20752/) (2025)

## Built on by (most cited first, 3 of 3)

- [A Survey on Rubric-Guided Reinforcement Learning for Language Models](https://mlsanity.com/paper/2608.27505/) (2026)
- [Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation](https://mlsanity.com/paper/2604.04532/) (2026)
- [Rank Reversal in Multilingual LLM Judges: A Label-Free Double-Centering Calibrator](https://mlsanity.com/paper/2608.22432/) (2026)

## Abstract (arXiv)

As large language models are increasingly deployed across diverse real-world applications, extending automated evaluation beyond English has become a critical challenge. Existing evaluation approaches are predominantly English-focused, and adapting them to other languages is hindered by the scarcity and cost of human-annotated judgments in most languages. We introduce a decomposition-based evaluation framework built around a Universal Criteria Set (UCS). UCS consists of a shared, language-agnostic set of evaluation dimensions, producing an interpretable intermediate representation that supports cross-lingual transfer with minimal supervision. Experiments on multiple faithfulness tasks across languages and model backbones demonstrate consistent improvements over strong baselines without requiring target-language annotations.