mlsanityData to 7 Oct 2026

SimGym: Traffic-Grounded Browser Agents for Offline A/B Testing in E-Commerce

arXiv 2602.01443 ↗First posted 1 Feb 2026cs.AIcited by 4

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

A/B testing remains the gold standard for evaluating e-commerce UI changes, yet it diverts traffic, takes weeks to reach significance, and risks harming user experience. We introduce SimGym, a scalable system for rapid offline A/B testing using traffic-grounded synthetic buyers powered by Large Language Model agents operating in a live browser. SimGym extracts per-shop buyer profiles and intents from production interaction data, identifies distinct behavioral archetypes, and simulates cohort-weighted sessions across control and treatment storefronts. We validate SimGym against real human outcomes from real UI changes on a major e-commerce platform under confounder control. Even without alignment post training, SimGym agents achieve state of the art alignment with observed outcome shifts and reduces experiment cycles from weeks to under an hour , enabling rapid experimentation without exposure to real buyers.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 24 of its 27 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

24 of 27 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025See, Think, Act: Online Shopper Behavior Simulation with VLM Agents22 Oct
  2. 2025LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?25 Sep
  3. 2025WebMall -- A Multi-Shop Benchmark for Evaluating Web Agents18 Aug
  4. 2025ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents6 Aug
  5. 2025DeepShop: A Benchmark for Deep Research Shopping Agents3 Jun
  6. 2025PAARS: Persona Aligned Agentic Retail Shoppers31 Mar
  7. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 17

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  2. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  4. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  5. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  6. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
  7. 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 119
  8. 2023On Generative Agents in Recommendationcited by 88
  9. 2024Large Language Models Empowered Personalized Web Agentscited by 64
  10. 2025Fara-7B: An Efficient Agentic Model for Computer Usecited by 34
  11. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 26
  12. 2024ChatShop: Interactive Information Seeking with Language Agentscited by 21
5 more
  1. 2025AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agentscited by 19
  2. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  3. 2024LMAgent: A Large-scale Multimodal Agents Society for Multi-user Simulationcited by 14
  4. 2024CXSimulator: A User Behavior Simulation using LLM Embeddings for Web-Marketing Campaign Assessmentcited by 8
  5. 2025ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?cited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agentscited by 4
  2. 2026Can AI Agents Simulate A/B Test Outcomes? A Validation Framework for Agentic Experimentationcited by 0
  3. 2026Data-Driven Persona-Conditioned Agents for A/B Test Simulationcited by 0
  4. 2026Augmented Hypothesis Testing with Persona-Based LLM Simulationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SimGym: A Framework for A/B Test Simulation in E-Commerce with Traffic-Grounded VLM Agents
  2. 2026MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operations
  3. 2026SimPersona: Learning Discrete Buyer Personas from Raw Clickstreams for Grounded E-Commerce Agents
  4. 2026Iterating Toward Better Search: A Two-Agent Simulation Framework for Evaluating Agentic Search Architectures in E-Commerce
  5. 2026EComStage: Stage-wise and Orientation-specific Benchmarking for Large Language Models in E-commerce
  6. 2026SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling
  7. 2026EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent
  8. 2026VISTA: A Versatile Interactive User Simulation Toolkit for Agent Evaluation