# Data-Driven Persona-Conditioned Agents for A/B Test Simulation

arXiv: https://arxiv.org/abs/2609.01038 · HTML: https://mlsanity.com/paper/2609.01038/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 12 of 12)

- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [Finetuning LLMs for Human Behavior Prediction in Social Science Experiments](https://mlsanity.com/paper/2509.05830/) (2025)
- [Mixture-of-Personas Language Models for Population Simulation](https://mlsanity.com/paper/2504.05019/) (2025)
- [PAARS: Persona Aligned Agentic Retail Shoppers](https://mlsanity.com/paper/2503.24228/) (2025)
- [Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs](https://mlsanity.com/paper/2603.00024/) (2026)
- [SimGym: Traffic-Grounded Browser Agents for Offline A/B Testing in E-Commerce](https://mlsanity.com/paper/2602.01443/) (2026)

## Built on by (most cited first, 1 of 1)

- [Data-Driven Personas for Survey Simulation: Insights into Simulation Alignment Across Data-Access Regimes](https://mlsanity.com/paper/2610.05828/) (2026)

## Abstract (arXiv)

A/B testing is the gold standard for evaluating product changes, but each experiment requires real user traffic, engineering effort, and weeks of measurement. We propose a simulation framework that predicts A/B test outcomes using LLM-powered agents conditioned on data-driven personas grounded in real user behavioral signals. Unlike prior work that relies on synthetic or rule-based personas, our agents are constructed from anonymized behavioral data - activity patterns, engagement signals, and inferred demographics - enabling more faithful population modeling. We frame A/B test simulation as a structured question task and systematically study (i) question design formats, (ii) the impact of persona data source and domain alignment, (iii) the trade-off between per-persona behavioral depth and population diversity, and (iv) efficient population subsampling. On a benchmark of 40 A/B tests spanning two metric types, our best configuration achieves 0.75-0.80 directional accuracy depending on the test metric, demonstrating that data-driven personas are a viable path toward fast, low-cost experiment pre-screening.