# Deliberate Lab: A Platform for Real-Time Human-AI Social Experiments

arXiv: https://arxiv.org/abs/2510.13011 · HTML: https://mlsanity.com/paper/2510.13011/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 15 of 15)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [Systematic Biases in LLM Simulations of Debates](https://mlsanity.com/paper/2402.04049/) (2024)
- [Evaluating Human-Language Model Interaction](https://mlsanity.com/paper/2212.09746/) (2022)
- [Opportunities and Risks of LLMs for Scalable Deliberation with Polis](https://mlsanity.com/paper/2306.11932/) (2023)
- [Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study](https://mlsanity.com/paper/2404.13414/) (2024)
- [Empirica: a virtual lab for high-throughput macro-level experiments](https://mlsanity.com/paper/2006.11398/) (2020)
- [Take It, Leave It, or Fix It: Measuring Productivity and Trust in Human-AI Collaboration](https://mlsanity.com/paper/2402.18498/) (2024)
- [Group Communication Analysis: A Computational Linguistics Approach for Detecting Sociocognitive Roles in Multi-Party Interactions](https://mlsanity.com/paper/1801.03563/) (2018)
- [AgentGroupChat: An Interactive Group Chat Simulacra For Better Eliciting Emergent Behavior](https://mlsanity.com/paper/2403.13433/) (2024)
- [SAUCE: Synchronous and Asynchronous User-Customizable Environment for Multi-Agent LLM Interaction](https://mlsanity.com/paper/2411.03397/) (2024)
- [Mitigating Barriers to Public Social Interaction with Meronymous Communication](https://mlsanity.com/paper/2402.17847/) (2024)

## Built on by (most cited first, 3 of 3)

- [Evaluating Language Models for Harmful Manipulation](https://mlsanity.com/paper/2603.25326/) (2026)
- [VIRENA: Virtual Arena for Research, Education, and Democratic Innovation](https://mlsanity.com/paper/2602.12207/) (2026)
- [Pairit: A Platform for Live Experiments on Human-AI Collaboration](https://mlsanity.com/paper/2609.09789/) (2026)

## Abstract (arXiv)

Social and behavioral scientists increasingly aim to study how humans interact, collaborate, and make decisions alongside artificial intelligence. However, the experimental infrastructure for such work remains underdeveloped: (1) few platforms support real-time, multi-party studies at scale; (2) most deployments require bespoke engineering, limiting replicability and accessibility, and (3) existing tools do not treat AI agents as first-class participants. We present Deliberate Lab, an open-source platform for large-scale, real-time behavioral experiments that supports both human participants and large language model (LLM)-based agents. We report on a 12-month public deployment of the platform (N=88 experimenters, N=9195 experiment participants), analyzing usage patterns and workflows. Case studies and usage scenarios are aggregated from platform users, complemented by in-depth interviews with select experimenters. By lowering technical barriers and standardizing support for hybrid human-AI experimentation, Deliberate Lab expands the methodological repertoire for studying collective decision-making and human-centered AI.