# Characterizing Web Search in The Age of Generative AI

arXiv: https://arxiv.org/abs/2510.11560 · HTML: https://mlsanity.com/paper/2510.11560/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 51 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 21 of 21)

- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios](https://mlsanity.com/paper/2401.17167/) (2024)
- [Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM](https://mlsanity.com/paper/2404.12259/) (2024)
- [Search Arena: Analyzing Search-Augmented LLMs](https://mlsanity.com/paper/2506.05334/) (2025)
- [Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger](https://mlsanity.com/paper/2502.12961/) (2025)
- [Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community](https://mlsanity.com/paper/2307.09751/) (2023)
- [Generative AI Search Engines as Arbiters of Public Knowledge: An Audit of Bias and Authority](https://mlsanity.com/paper/2405.14034/) (2024)
- [Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics](https://mlsanity.com/paper/2010.14531/) (2020)
- [Evaluating Generative Ad Hoc Information Retrieval](https://mlsanity.com/paper/2311.04694/) (2023)
- [Open-World Evaluation for Retrieving Diverse Perspectives](https://mlsanity.com/paper/2409.18110/) (2024)
- [Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions](https://mlsanity.com/paper/2403.12077/) (2024)
- [AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs](https://mlsanity.com/paper/2410.19692/) (2024)
- [NExT-Search: Rebuilding User Feedback Ecosystem for Generative AI Search](https://mlsanity.com/paper/2505.14680/) (2025)
- [What Users See - Structures in Search Engine Results Pages](https://mlsanity.com/paper/1511.05802/) (2015)
- [Auditing Source Diversity Bias in Video Search Results Using Virtual Agents](https://mlsanity.com/paper/2106.02715/) (2021)
- [A Framework for Evaluating the Retrieval Effectiveness of Search Engines](https://mlsanity.com/paper/1511.05817/) (2015)
- [Taxonomy and Analysis of Sensitive User Queries in Generative AI Search](https://mlsanity.com/paper/2404.08672/) (2024)

## Built on by (most cited first, 5 of 5)

- [To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling](https://mlsanity.com/paper/2605.00737/) (2026)
- [Generative Engine Optimization at Scale: Measuring Brand Visibility Across AI Search Engines](https://mlsanity.com/paper/2606.20065/) (2026)
- [How Large Language Models Source Brand Reputation Across Languages and Markets](https://mlsanity.com/paper/2606.25787/) (2026)
- [GEO-Flag: Detecting and Measuring GEO-Optimized Web Content](https://mlsanity.com/paper/2608.16824/) (2026)
- [When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Models](https://mlsanity.com/paper/2608.19230/) (2026)

## Abstract (arXiv)

The advent of LLMs has given rise to generative search, a new search paradigm in which LLMs retrieve information from the web related to a query and synthesize it into a single, coherent response. This paradigm differs fundamentally from traditional web search, where results are returned as a ranked list of independent web pages. In this paper, we ask: Along what dimensions does generative search differ from traditional search? We conduct a systematic comparison between Google organic search and five generative search systems from three providers: Google, OpenAI, and Perplexity. Our analysis reveals substantial variation among engines in their reliance on internal v.s. external knowledge, source diversity, and stability. While generative systems often achieve topical coverage comparable to traditional search, they do so using markedly different retrieval footprints and synthesis strategies. We further show that the outputs of generative search can vary across time and executions, raising new challenges for robustness. Our findings demonstrate that generative search introduces new dimensions that are not captured by existing evaluation paradigms, motivating the development of evaluations that explicitly account for retrieval behavior, synthesis, and stability in generative search systems.