mlsanityData to 14 Oct 2026

CLEV: LLM-Based Evaluation Through Lightweight Efficient Voting for Free-Form Question-Answering

arXiv 2503.08542 ↗First posted 11 Mar 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Evaluating free-form Question Answering (QA) remains a challenge due to its diverse and open-ended nature. Traditional automatic metrics fail to capture semantic equivalence or accommodate the variability of open-ended responses. Leveraging Large Language Models (LLMs) as evaluators offers a promising alternative due to their strong language understanding and instruction-following capabilities. We propose Consensus via Lightweight Efficient Voting (CLEV), which employs two primary LLMs as judges and invokes a third judge only in cases of disagreement. This approach prioritizes evaluation reliability while reducing unnecessary computational demands. Through experiments, including human evaluation, we demonstrate CLEV's ability to provide consistent, scalable, and resource-efficient assessments, establishing it as a robust framework for evaluating LLMs on free-form QA.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

32 of 48 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  11. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  12. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
20 more
  1. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  2. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  3. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  4. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  5. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  6. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  7. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
  8. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  9. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  10. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 223
  11. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 151
  12. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 105
  13. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 88
  14. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  15. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 54
  16. 2024Are LLM-based Evaluators Confusing NLG Quality Criteria?cited by 36
  17. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  18. 2024Quantifying the Capabilities of LLMs across Scale and Precisioncited by 19
  19. 2024How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs?cited by 13
  20. 2024Latent Concept-based Explanation of NLP Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026SCOPE: Selective Conformal Optimized Pairwise LLM Judgingcited by 14
  2. 2025SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QAcited by 2
  3. 2026Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guaranteescited by 2
  4. 2026Margin-Adaptive Confidence Ranking for Reliable LLM Judgementcited by 1
  5. 2025Integrated Framework for LLM Evaluation with Answer Generationcited by 0
  6. 2026What Do Rationales Communicate? A Message-Intervention Study in Role-Specialized QAcited by 0