# What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce

arXiv: https://arxiv.org/abs/2508.02630 · HTML: https://mlsanity.com/paper/2508.02630/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 25 of 25)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale](https://mlsanity.com/paper/2409.08264/) (2024)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents](https://mlsanity.com/paper/2504.00906/) (2025)
- [A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond](https://mlsanity.com/paper/2503.21614/) (2025)
- [The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use](https://mlsanity.com/paper/2411.10323/) (2024)
- [PersonalLLM: Tailoring LLMs to Individual Preferences](https://mlsanity.com/paper/2409.20296/) (2024)
- [VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks](https://mlsanity.com/paper/2401.13649/) (2024)
- [eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data](https://mlsanity.com/paper/2402.08831/) (2024)
- [DeepShop: A Benchmark for Deep Research Shopping Agents](https://mlsanity.com/paper/2506.02839/) (2025)
- [The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective](https://mlsanity.com/paper/2506.04301/) (2025)
- [Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models](https://mlsanity.com/paper/2410.20745/) (2024)
- [Large Language Models for Market Research: A Data-augmentation Approach](https://mlsanity.com/paper/2412.19363/) (2024)
- [Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC](https://mlsanity.com/paper/2506.24045/) (2025)
- [LiLiuM: eBay's Large Language Models for e-commerce](https://mlsanity.com/paper/2406.12023/) (2024)
- [PUMGPT: A Large Vision-Language Model for Product Understanding](https://mlsanity.com/paper/2308.09568/) (2023)
- [Designing Algorithmic Delegates: The Role of Indistinguishability in Human-AI Handoff](https://mlsanity.com/paper/2506.03102/) (2025)
- [Social Status and Badge Design](https://mlsanity.com/paper/1312.2299/) (2013)
- [Harnessing Natural Experiments to Quantify the Causal Effect of Badges](https://mlsanity.com/paper/1707.08160/) (2017)
- [These Deals Won't Last! Longevity, Uniformity and Bias in Product Badge Assignment in E-Commerce Platforms](https://mlsanity.com/paper/2204.12552/) (2022)

## Built on by (most cited first, 22 of 22)

- [Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets](https://mlsanity.com/paper/2510.25779/) (2025)
- [Auditing Preferences for Brands and Cultures in LLMs](https://mlsanity.com/paper/2603.18300/) (2026)
- [Would a Large Language Model Pay Extra for a View? Inferring Willingness to Pay from Subjective Choices](https://mlsanity.com/paper/2602.09802/) (2026)
- [Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization](https://mlsanity.com/paper/2604.09574/) (2026)
- [Strategic Buying Agents](https://mlsanity.com/paper/2607.04708/) (2026)
- [Optimizing Service Operations via LLM-Powered Multi-Agent Simulation](https://mlsanity.com/paper/2604.04383/) (2026)
- [Generative AI Advertising as a Problem of Trustworthy Commercial Intervention](https://mlsanity.com/paper/2605.18673/) (2026)
- [Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce](https://mlsanity.com/paper/2608.02441/) (2026)
- [Agent-Facing Information Design in LLM Tool Registries](https://mlsanity.com/paper/2605.23916/) (2026)
- [Paying to Know: Micro-Transaction Markets for Verified Product Information in Agentic E-Commerce](https://mlsanity.com/paper/2606.24783/) (2026)
- [REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editing](https://mlsanity.com/paper/2607.05364/) (2026)
- [The User Asks, Platforms Compete: How Agentic Recommendation Markets Take Shape](https://mlsanity.com/paper/2607.25253/) (2026)
- [From Passive Delegates to Strategic Negotiators: Reinforcing Social Reasoning in Small Language Models with SocialRL](https://mlsanity.com/paper/2608.13787/) (2026)
- [A Formal Analysis of Agent Payment Protocols](https://mlsanity.com/paper/2609.00060/) (2026)
- ["If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations](https://mlsanity.com/paper/2609.18729/) (2026)
- [Multi-Dimensional Matching](https://mlsanity.com/paper/2609.29958/) (2026)
- [Selective Elicitation as a Commercial Influence Channel: A Reproducible Synthetic Shopping-Agent Stress Test](https://mlsanity.com/paper/2609.36614/) (2026)
- [SoK: Decentralized Agent Economic Infrastructure](https://mlsanity.com/paper/2610.01756/) (2026)
- [Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It](https://mlsanity.com/paper/2610.03195/) (2026)
- [AI-Mediation in Retail Markets: Revenue Effects and Assortment Optimization](https://mlsanity.com/paper/2610.12917/) (2026)
- [Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions](https://mlsanity.com/paper/2610.14334/) (2026)
- [Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commerce](https://mlsanity.com/paper/2610.17378/) (2026)

## Abstract (arXiv)

Online marketplaces will be transformed by autonomous AI agents acting on behalf of consumers. Rather than humans browsing and clicking, AI agents can parse webpages or leverage APIs to view, evaluate and choose products. We investigate the behavior of AI agents using ACES, a provider-agnostic framework for auditing agent decision-making. We reveal that agents can exhibit choice homogeneity, often concentrating demand on a few ``modal'' products while ignoring others entirely. Yet, these preferences are unstable: model updates can drastically reshuffle market shares. Furthermore, randomized trials show that while agents have improved over time on simple tasks with a clearly identified best choice, they exhibit strong position biases -- varying across providers and model versions, and persisting even in text-only "headless" interfaces -- undermining any universal notion of a ``top'' rank. Agents also consistently penalize sponsored tags while rewarding platform endorsements, and sensitivities to price, ratings, and reviews vary sharply across models. Finally, we demonstrate that sellers can respond: a seller-side agent making simple, query-conditional description tweaks can drive significant gains in market share. These findings reveal that agentic markets are volatile and fundamentally different from human-centric commerce, highlighting the need for continuous auditing and raising questions for platform design, seller strategy and regulation.