# How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews

arXiv: https://arxiv.org/abs/2604.27790 · HTML: https://mlsanity.com/paper/2604.27790/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 62 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search](https://mlsanity.com/paper/2206.06588/) (2022)
- [Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation](https://mlsanity.com/paper/1806.01156/) (2018)
- [GEO: Generative Engine Optimization](https://mlsanity.com/paper/2311.09735/) (2023)
- [Neural Retrievers are Biased Towards LLM-Generated Content](https://mlsanity.com/paper/2310.20501/) (2023)
- [Ranking Manipulation for Conversational Search Engines](https://mlsanity.com/paper/2406.03589/) (2024)
- [Measuring Personalization of Web Search](https://mlsanity.com/paper/1706.05011/) (2017)
- [GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search](https://mlsanity.com/paper/2412.20953/) (2024)
- [Generative Engine Optimization: How to Dominate AI Search](https://mlsanity.com/paper/2509.08919/) (2025)
- [Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Images](https://mlsanity.com/paper/2311.14084/) (2023)
- [NExT-Search: Rebuilding User Feedback Ecosystem for Generative AI Search](https://mlsanity.com/paper/2505.14680/) (2025)
- [In a Few Words: Comparing Weak Supervision and LLMs for Short Query Intent Classification](https://mlsanity.com/paper/2504.21398/) (2025)
- [Auditing LLM Editorial Bias in News Media Exposure](https://mlsanity.com/paper/2510.27489/) (2025)

## Built on by (most cited first, 2 of 2)

- [GEO-Flag: Detecting and Measuring GEO-Optimized Web Content](https://mlsanity.com/paper/2608.16824/) (2026)
- [Measuring GEO Visibility: Prompt Corpora Define the Answer Market](https://mlsanity.com/paper/2609.06811/) (2026)

## Abstract (arXiv)

Generative AI is being increasingly integrated into web search for the convenience it provides users. In this work, we aim to understand how generative AI disrupts web search by retrieving and presenting the information and sources differently from traditional search engines. We introduce a public benchmark dataset of 11,500 user queries to support our study and future research of generative search. We compare the search results returned by Google's search engine, the accompanying AI Overview (AIO), and Gemini Flash 2.5 for each query. We have made several key findings. First, we find that for 51.5% of representative, real-user queries, AIOs are generated, and are displayed above the organic search results. Controversial questions frequently result in an AIO. Second, we show that the retrieved sources are substantially different for each search engine (<0.2 average Jaccard similarity). Traditional Google search is significantly more likely to retrieve information from popular or institutional websites in government or education, while generative search engines are significantly more likely to retrieve Google-owned content. Third, we observe that websites that block Google's AI crawler are significantly less likely to be retrieved by AIOs, despite having access to the content. Finally, AIOs are less consistent when processing two runs of the same query, and are less robust to minor query edits. Our findings have important implications for understanding how generative search impacts website visibility, the effectiveness of generative engine optimization techniques, and the information users receive. We call for revenue frameworks to foster a sustainable and mutually beneficial ecosystem for publishers and generative search providers.