mlsanityData to 14 Oct 2026

Auto-ARGUE: LLM-Based Report Generation Evaluation

arXiv 2509.26184 ↗First posted 30 Sep 2025cs.IR cs.AI cs.CLcited by 14

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Generation of citation-backed reports is a primary use case for retrieval-augmented generation (RAG) systems. While open-source evaluation tools exist for various RAG tasks, tools designed for report generation are lacking. Accordingly, we introduce Auto-ARGUE, a robust LLM-based implementation of the recently proposed ARGUE framework for report generation evaluation. We present analysis of Auto-ARGUE on the report generation pilot task from the TREC 2024 NeuCLIR track and on two tasks from the TREC 2024 RAG track, showing good system-level correlations with human judgments. Additionally, we release ARGUE-Viz, a web app for visualization and fine-grained analysis of Auto-ARGUE judgments and scores.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 17 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

10 of 17 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Models21 Apr
  2. 2025GINGER: Grounded Information Nugget-Based Generation of Responses23 Mar
  3. 2024Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework14 Nov
  4. 2024Generative Information Retrieval Evaluation11 Apr
  5. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov
  6. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 4

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  3. 2024On the Evaluation of Machine-Generated Reportscited by 23
  4. 2025Overview of the TREC 2024 NeuCLIR Trackcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025Principled Context Engineering for RAG: Statistical Guarantees via Conformal Predictioncited by 6
  2. 2025Query Decomposition for RAG: Balancing Exploration-Exploitationcited by 5
  3. 2026Overview of the TREC 2025 RAGTIME Trackcited by 5
  4. 2026Incorporating Q&A Nuggets into Retrieval-Augmented Generationcited by 2
  5. 2026LANCER: LLM Reranking for Nugget Coveragecited by 2
  6. 2026CoverageBench: Evaluating Information Coverage across Tasks and Domainscited by 2
  7. 2025Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generationcited by 1
  8. 2026Insider Knowledge: How Much Can RAG Systems Gain from Evaluation Secrets?cited by 1
  9. 2026Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coveragecited by 1
  10. 2026DoGMaTiQ: Automated Generation of Question-and-Answer Nuggets for Report Evaluationcited by 1
  11. 2026Supporting Humans in Evaluating AI Summaries of Legal Depositionscited by 0
  12. 2026Distributed Generative Inference of LLM at Internet Scales with Multi-Dimensional Communication Optimizationcited by 0
2 more
  1. 2026Retrieval, not hallucinations, will be the limiting factor for LLM-based clinical AI toolscited by 0
  2. 2026Knowledge Pull Requests for Continual Document Authoringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Capacity of Citation Generation by Large Language Models
  2. 2025A Systematic Literature Review of Retrieval-Augmented Generation: Techniques, Metrics, and Challenges
  3. 2025OmniBench-RAG: A Multi-Domain Evaluation Platform for Retrieval-Augmented Generation Tools
  4. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  5. 2024Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation
  6. 2025AutoSurvey2: Empowering Researchers with Next Level Automated Literature Surveys
  7. 2025VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verification
  8. 2025RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation