mlsanityData to 14 Oct 2026

Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses

arXiv 2410.22349 ↗First posted 15 Oct 2024cs.IR cs.AI cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Large Language Model (LLM)-based applications are graduating from research prototypes to products serving millions of users, influencing how people write and consume information. A prominent example is the appearance of Answer Engines: LLM-based generative search engines supplanting traditional search engines. Answer engines not only retrieve relevant sources to a user query but synthesize answer summaries that cite the sources. To understand these systems' limitations, we first conducted a study with 21 participants, evaluating interactions with answer vs. traditional search engines and identifying 16 answer engine limitations. From these insights, we propose 16 answer engine design recommendations, linked to 8 metrics. An automated evaluation implementing our metrics on three popular engines (You.com, Perplexity.ai, BingChat) quantifies common limitations (e.g., frequent hallucination, inaccurate citation) and unique features (e.g., variation in answer confidence), with results mirroring user study insights. We release our Answer Engine Evaluation benchmark (AEE) to facilitate transparent evaluation of LLM-based applications.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 24 of its 76 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

24 of 76 references.

References · 24

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  2. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  4. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  5. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  6. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  7. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  8. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
  9. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  10. 2024RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculturecited by 81
  11. 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
  12. 2024FABLES: Evaluating faithfulness and content selection in book-length summarizationcited by 65
12 more
  1. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Frameworkcited by 39
  2. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 28
  3. 2024Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approachcited by 26
  4. 2023Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Modelscited by 20
  5. 2024Evaluation of RAG Metrics for Question Answering in the Telecom Domaincited by 20
  6. 2023Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articlescited by 16
  7. 2024From Melting Pots to Misrepresentations: Exploring Harms in Generative AIcited by 13
  8. 2023The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysiscited by 12
  9. 2023Evaluating Large Language Models for Health-related Queries with Presuppositionscited by 12
  10. 2023Towards a Holistic Approach: Understanding Sociodemographic Biases in NLP Models using an Interdisciplinary Lenscited by 9
  11. 2024Search Engines Post-ChatGPT: How Generative Artificial Intelligence Could Make Search Less Reliablecited by 6
  12. 2021Axes for Sociotechnical Inquiry in AI Researchcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  2. 2025Open Deep Search: Democratizing Search with Open-source Reasoning Agentscited by 49
  3. 2025Modifying Large Language Model Post-Training for Diverse Creative Writingcited by 40
  4. 2025Supervising the search process produces reliable and generalizable information-seeking agentscited by 20
  5. 2025GRADA: Graph-based Reranking against Adversarial Documents Attackcited by 10
  6. 2025Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systemscited by 10
  7. 2025Retrieval-augmented systems can be dangerous medical communicatorscited by 5
  8. 2025Orca: Browsing at Scale Through User-Driven and AI-Facilitated Orchestration Across Malleable Webpagescited by 5
  9. 2025Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citationcited by 4
  10. 2025Attacks by Content: Automated Fact-checking is an AI Security Issuecited by 4
  11. 2025Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answerscited by 3
  12. 2026Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impactcited by 1
3 more
  1. 2026Agents in the Wild: Where Research Meets Deploymentcited by 0
  2. 2026Redesigning and Auditing Deep Research Writing for Faithful Reportscited by 0
  3. 2026Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answeringcited by 0