mlsanityData to 14 Oct 2026

Plans for Evaluating Structured Generative Search Summaries

arXiv 2605.26400 ↗First posted 26 May 2026cs.IR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a framework for evaluating structured generative search summaries that are placed atop organic web search results. A structured summary, generated by a large language model, typically consists of an overview, several sections with section titles, and a list of source documents that are cited within the summary. We then describe our plans for implementing and evaluating the framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 56 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

21 of 56 references.

References · 21

  1. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  2. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  3. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  4. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  5. 2022Teaching language models to support answers with verified quotescited by 292
  6. 2021Measuring Attribution in Natural Language Generation Modelscited by 256
  7. 2025Why Language Models Hallucinatecited by 220
  8. 2023Perspectives on Large Language Models for Relevance Judgmentcited by 85
  9. 2023Context-faithful Prompting for Large Language Modelscited by 80
  10. 2024LLM-based relevance assessment still can't replace human relevance assessmentcited by 27
  11. 2024Don't Use LLMs to Make Relevance Judgmentscited by 26
  12. 2025The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Modelscited by 24
9 more
  1. 2024A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Lookcited by 18
  2. 2025GINGER: Grounded Information Nugget-Based Generation of Responsescited by 16
  3. 2024Verifiable Generation with Subsentence-Level Fine-Grained Citationscited by 15
  4. 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Stylecited by 11
  5. 2025Overview of the TREC 2023 deep learning trackcited by 9
  6. 2025Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggetscited by 8
  7. 2023Overview of the TREC 2021 Fair Ranking Trackcited by 7
  8. 2022A Versatile Framework for Evaluating Ranked Lists in terms of Group Fairness and Relevancecited by 4
  9. 2025FACE: A Fine-Grained Reference-Free Evaluator for Conversational Information Accesscited by 2

Built on by

Not cited yet.