Rate, Explain and Cite (REC): Enhanced Explanation and Attribution in Automatic Evaluation by Large Language Models
Not on the map. Why?
Abstract (arXiv)
LLMs have demonstrated impressive proficiency in generating coherent and high-quality text, making them valuable across a range of text-generation tasks. However, rigorous evaluation of this generated content is crucial, as ensuring its quality remains a significant challenge due to persistent issues such as factual inaccuracies and hallucination. This paper introduces three fine-tuned general-purpose LLM autoevaluators, REC-8B, REC-12B and REC-70B, specifically designed to evaluate generated text across several dimensions: faithfulness, instruction following, coherence, and completeness. These models not only provide ratings for these metrics but also offer detailed explanation and verifiable citation, thereby enhancing trust in the content. Moreover, the models support various citation modes, accommodating different requirements for latency and granularity. Extensive evaluations on diverse benchmarks demonstrate that our general-purpose LLM auto-evaluator, REC-70B, outperforms state-of-the-art LLMs, excelling in content evaluation by delivering better quality explanation and citation with minimal bias. Our REC dataset and models are available at https://github.com/adelaidehsu/REC.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 3 Nov 2024.
Builds on
48 of 58 references.
References · 48
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
36 more
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
- 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
- 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
- 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2019ELI5: Long Form Question Answeringcited by 567
- 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
- 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
- 2023GPTScore: Evaluate as You Desirecited by 357
- 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
- 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 314
- 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
- 2023Evaluating Verifiability in Generative Search Enginescited by 297
- 2022Teaching language models to support answers with verified quotescited by 290
- 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
- 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
- 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
- 2023Generative Judge for Evaluating Alignmentcited by 163
- 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 147
- 2023Shepherd: A Critic for Language Model Generationcited by 96
- 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
- 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
- 2021Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systemscited by 73
- 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
- 2023TIGERScore: Towards Building Explainable Metric for All Text Generation Taskscited by 69
- 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
- 2023PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptionscited by 53
- 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
- 2023Towards Verifiable Text Generation with Evolving Memory and Self-Reflectioncited by 21
- 2024Verifiable Generation with Subsentence-Level Fine-Grained Citationscited by 15
Built on by
Citing papers per year: 2025: 1.
1 paper.