# Auditing Google's AI Overviews and Featured Snippets: A Case Study on Baby Care and Pregnancy

arXiv: https://arxiv.org/abs/2511.12920 · HTML: https://mlsanity.com/paper/2511.12920/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 78 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Builds on (most cited first, 16 of 16)

- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation](https://mlsanity.com/paper/2509.08825/) (2025)
- [Ask the Right Questions: Active Question Reformulation with Reinforcement Learning](https://mlsanity.com/paper/1705.07830/) (2017)
- [ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search](https://mlsanity.com/paper/2006.05324/) (2020)
- [Logical Consistency of Large Language Models in Fact-checking](https://mlsanity.com/paper/2412.16100/) (2024)
- [Generative AI Search Engines as Arbiters of Public Knowledge: An Audit of Bias and Authority](https://mlsanity.com/paper/2405.14034/) (2024)
- [How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions](https://mlsanity.com/paper/1911.09247/) (2019)
- [Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions](https://mlsanity.com/paper/2403.12077/) (2024)
- [How does AI chat change search behaviors?](https://mlsanity.com/paper/2307.03826/) (2023)
- [The Evolution of Web Search User Interfaces -- An Archaeological Analysis of Google Search Engine Result Pages](https://mlsanity.com/paper/2301.08613/) (2023)
- [Human Trust in AI Search: A Large-Scale Experiment](https://mlsanity.com/paper/2504.06435/) (2025)
- [Reduced AI Acceptance After the Generative AI Boom: Evidence From a Two-Wave Survey Study](https://mlsanity.com/paper/2510.23578/) (2025)
- [Featured Snippets Results in Google Web Search: An Exploratory Study](https://mlsanity.com/paper/1907.04891/) (2019)
- [Searching for Representation: A sociotechnical audit of googling for members of U.S. Congress](https://mlsanity.com/paper/2109.07012/) (2021)

## Built on by (most cited first, 7 of 7)

- [Answer Bubbles: Information Exposure in AI-Mediated Search](https://mlsanity.com/paper/2603.16138/) (2026)
- [Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement](https://mlsanity.com/paper/2603.08924/) (2026)
- [Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact](https://mlsanity.com/paper/2605.14021/) (2026)
- [Who Gets Named: Citation Type Predicts Individual Naming by Grounded Language Models, and a Roster Instrument Captures 0.5% of It](https://mlsanity.com/paper/2607.23893/) (2026)
- [WildSEEK: Evaluating Language Models for Information-Seeking](https://mlsanity.com/paper/2608.30683/) (2026)
- [API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces](https://mlsanity.com/paper/2609.08861/) (2026)
- ["If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations](https://mlsanity.com/paper/2609.18729/) (2026)

## Abstract (arXiv)

Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no control over their presentation. Through a systematic algorithm audit of 1,508 real baby care and pregnancy-related queries, we evaluate the quality and consistency of these information displays. Our robust evaluation framework assesses multiple quality dimensions, including answer consistency, relevance, presence of medical safeguards, source categories, and sentiment alignment. Our results reveal concerning gaps in information consistency, with information in AIO and FS displayed on the same search result page being inconsistent with each other in 33% of cases. Despite high relevance scores, both features critically lack medical safeguards (present in just 11% of AIO and 7% of FS responses). While health and wellness websites dominate source categories for both, AIO and FS, FS also often link to commercial sources. These findings have important implications for public health information access and demonstrate the need for stronger quality controls in AI-mediated health information. Our methodology provides a transferable framework for auditing AI systems across high-stakes domains where information quality directly impacts user well-being.