# SimGym: Traffic-Grounded Browser Agents for Offline A/B Testing in E-Commerce

arXiv: https://arxiv.org/abs/2602.01443 · HTML: https://mlsanity.com/paper/2602.01443/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 27 references that are in the map.
- First posted on arXiv 1 Feb 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 24 of 24)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024)
- [Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents](https://mlsanity.com/paper/2411.06559/) (2024)
- [WebCanvas: Benchmarking Web Agents in Online Environments](https://mlsanity.com/paper/2406.12373/) (2024)
- [On Generative Agents in Recommendation](https://mlsanity.com/paper/2310.10108/) (2023)
- [Large Language Models Empowered Personalized Web Agents](https://mlsanity.com/paper/2410.17236/) (2024)
- [DeepShop: A Benchmark for Deep Research Shopping Agents](https://mlsanity.com/paper/2506.02839/) (2025)
- [Fara-7B: An Efficient Agentic Model for Computer Use](https://mlsanity.com/paper/2511.19663/) (2025)
- [OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation](https://mlsanity.com/paper/2506.05606/) (2025)
- [ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents](https://mlsanity.com/paper/2508.04266/) (2025)
- [ChatShop: Interactive Information Seeking with Language Agents](https://mlsanity.com/paper/2404.09911/) (2024)
- [AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents](https://mlsanity.com/paper/2504.09723/) (2025)
- [Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning](https://mlsanity.com/paper/2507.17842/) (2025)
- [WebMall -- A Multi-Shop Benchmark for Evaluating Web Agents](https://mlsanity.com/paper/2508.13024/) (2025)
- [LMAgent: A Large-scale Multimodal Agents Society for Multi-user Simulation](https://mlsanity.com/paper/2412.09237/) (2024)
- [PAARS: Persona Aligned Agentic Retail Shoppers](https://mlsanity.com/paper/2503.24228/) (2025)
- [LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?](https://mlsanity.com/paper/2509.21501/) (2025)
- [ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?](https://mlsanity.com/paper/2511.22978/) (2025)
- [CXSimulator: A User Behavior Simulation using LLM Embeddings for Web-Marketing Campaign Assessment](https://mlsanity.com/paper/2407.21553/) (2024)
- [See, Think, Act: Online Shopper Behavior Simulation with VLM Agents](https://mlsanity.com/paper/2510.19245/) (2025)

## Built on by (most cited first, 4 of 4)

- [ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents](https://mlsanity.com/paper/2605.16116/) (2026)
- [Data-Driven Persona-Conditioned Agents for A/B Test Simulation](https://mlsanity.com/paper/2609.01038/) (2026)
- [Can AI Agents Simulate A/B Test Outcomes? A Validation Framework for Agentic Experimentation](https://mlsanity.com/paper/2608.02345/) (2026)
- [Augmented Hypothesis Testing with Persona-Based LLM Simulations](https://mlsanity.com/paper/2609.24629/) (2026)

## Abstract (arXiv)

A/B testing remains the gold standard for evaluating e-commerce UI changes, yet it diverts traffic, takes weeks to reach significance, and risks harming user experience. We introduce SimGym, a scalable system for rapid offline A/B testing using traffic-grounded synthetic buyers powered by Large Language Model agents operating in a live browser. SimGym extracts per-shop buyer profiles and intents from production interaction data, identifies distinct behavioral archetypes, and simulates cohort-weighted sessions across control and treatment storefronts. We validate SimGym against real human outcomes from real UI changes on a major e-commerce platform under confounder control. Even without alignment post training, SimGym agents achieve state of the art alignment with observed outcome shifts and reduces experiment cycles from weeks to under an hour , enabling rapid experimentation without exposure to real buyers.