mlsanityData to 7 Oct 2026

DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation

arXiv 2405.15329 ↗First posted 24 May 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

The acceleration of Large Language Models (LLMs) research has opened up new possibilities for evaluating generated texts. They serve as scalable and economical evaluators, but the question of how reliable these evaluators are has emerged as a crucial research question. Prior research efforts in the meta-evaluation of LLMs as judges limit the prompting of an LLM to a single use to obtain a final evaluation decision. They then compute the agreement between LLMs' outputs and human labels. This lacks interpretability in understanding the evaluation capability of LLMs. In light of this challenge, we propose Decompose and Aggregate, which breaks down the evaluation process into different stages based on pedagogical practices. Our experiments illustrate that it not only provides a more interpretable window for how well LLMs evaluate, but also leads to improvements up to 39.6% for different LLMs on a variety of meta-evaluation benchmarks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

25 of 43 references.

References · 25

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  5. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  6. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  7. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  8. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  9. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  10. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  11. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  12. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
13 more
  1. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  2. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  3. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
  4. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 146
  5. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 104
  6. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  7. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  8. 2024Investigating Data Contamination for Pre-training Language Modelscited by 91
  9. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  10. 2023Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoningcited by 49
  11. 2024Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debatecited by 40
  12. 2022On the Evaluation Metrics for Paraphrase Generationcited by 32
  13. 2023CoAScore: Chain-of-Aspects Prompting for NLG Evaluationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  2. 2025Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Predictioncited by 26
  3. 2026Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judgecited by 7
  4. 2026Hearing Between the Lines: Unlocking the Reasoning Power of LLMs for Speech Evaluationcited by 5
  5. 2025Checklist Engineering Empowers Multilingual LLM Judgescited by 3
  6. 2025COGENT: A Curriculum-oriented Framework for Generating Grade-appropriate Educational Contentcited by 2
  7. 2025A Dual-Perspective NLG Meta-Evaluation Framework with Automatic Benchmark and Better Interpretabilitycited by 1
  8. 2025KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narrationcited by 1
  9. 2026Do Audio Language Models Use Paralinguistic Evidence? Counterfactual Audits for Response Evaluationcited by 1
  10. 2026Towards a Reliable and Practical Eval Pipelinecited by 0
  11. 2026Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judgescited by 0
  12. 2026Designing Reliable LLM-as-a-Judge Measurement Systems for Multi-Turn Business Agentscited by 0