mlsanityData to 7 Oct 2026

Fusion-Eval: Integrating Assistant Evaluators with LLMs

arXiv 2311.09204 ↗First posted 15 Nov 2023cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Evaluating natural language systems poses significant challenges, particularly in the realms of natural language understanding and high-level reasoning. In this paper, we introduce 'Fusion-Eval', an innovative approach that leverages Large Language Models (LLMs) to integrate insights from various assistant evaluators. The LLM is given the example to evaluate along with scores from the assistant evaluators. Each of these evaluators specializes in assessing distinct aspects of responses. Fusion-Eval achieves a 0.962 system-level Kendall-Tau correlation with humans on SummEval and a 0.744 turn-level Spearman correlation on TopicalChat, which is significantly higher than baseline methods. These results highlight Fusion-Eval's significant potential in the realm of natural language system evaluation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 20 of its 51 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

20 of 51 references.

References · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  8. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  12. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
8 more
  1. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  2. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  3. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  4. 2023GPTScore: Evaluate as You Desirecited by 357
  5. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  6. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 186
  7. 2023SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluationcited by 42
  8. 2022SMART: Sentences as Basic Units for Text Evaluationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  2. 2024Can Large Language Models Beat Wall Street? Unveiling the Potential of AI in Stock Selectioncited by 29
  3. 2025Elementary Math Word Problem Generation using Large Language Modelscited by 0
  4. 2025Exploring the features used for summary evaluation by Human and GPTcited by 0
  5. 2026MILE-RefHumEval: A Reference-Free, Multi-Independent LLM Framework for Human-Aligned Evaluationcited by 0
  6. 2026Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judgescited by 0