# "If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations

arXiv: https://arxiv.org/abs/2609.18729 · HTML: https://mlsanity.com/paper/2609.18729/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 13 of 13)

- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment](https://mlsanity.com/paper/2307.03744/) (2023)
- [Large Language Models as Recommender Systems: A Study of Popularity Bias](https://mlsanity.com/paper/2406.01285/) (2024)
- [What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce](https://mlsanity.com/paper/2508.02630/) (2025)
- [E-GEO: A Testbed for Generative Engine Optimization in E-Commerce](https://mlsanity.com/paper/2511.20867/) (2025)
- [Auditing Google's AI Overviews and Featured Snippets: A Case Study on Baby Care and Pregnancy](https://mlsanity.com/paper/2511.12920/) (2025)
- [LLM Spirals of Delusion: A Benchmarking Audit Study of AI Chatbot Interfaces](https://mlsanity.com/paper/2604.06188/) (2026)
- [Auditing Preferences for Brands and Cultures in LLMs](https://mlsanity.com/paper/2603.18300/) (2026)
- [Tracking Conversations: Measuring Content and Identity Exposure on AI Chatbots](https://mlsanity.com/paper/2604.27438/) (2026)
- [Synthetic Sources?: Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources](https://mlsanity.com/paper/2605.23684/) (2026)
- [Persona Conditioning of Brand Recommendations in Retrieval-Augmented Commercial Chat: A Prominence-Stratified Cross-Provider Audit](https://mlsanity.com/paper/2605.30207/) (2026)

## Abstract (arXiv)

Consumers increasingly use AI chatbots for advice on what to buy. With companies like OpenAI and Google monetising their AI through advertising, this raises difficult questions about the bias and impartiality of such advice. In response, we conduct an AI audit of popular chatbots using real commercial-advice queries. First, we curate a dataset of 2,528 real commercial-advice queries (ConsumerQ). Then, we evaluate 1,536 responses to product queries from popular AI chatbots: ChatGPT (chatbot and API), Google Gemini (chatbot and API), and Google Search (AI Overviews). We find that ChatGPT expresses a first-person product preference in 79% of product-recommending responses, compared with 7% for Gemini and 2% for AI Overviews, while the products recommended often change across repeated requests. Displayed sources vary strongly: for the same query, the ChatGPT and Gemini interfaces share only 5.4% of domains on average, with no domain in common in 76.7% of comparisons. APIs provide a different view from their corresponding interfaces, with mean domain overlaps of 12.0% for ChatGPT and 14.8% for Gemini, and also differ in the types and layers of source information they expose. Our findings show that neither isolated responses nor API observations can be assumed to represent the commercial advice consumers encounter. Independent audits of AI-mediated commercial advice should therefore account for repeated responses, consumer-facing conditions, and the source layer being observed.