# Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Games

arXiv: https://arxiv.org/abs/2401.17435 · HTML: https://mlsanity.com/paper/2401.17435/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 76 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022)
- [Towards Making the Most of ChatGPT for Machine Translation](https://mlsanity.com/paper/2303.13780/) (2023)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study](https://mlsanity.com/paper/2304.04339/) (2023)
- [Training Question Answering Models From Synthetic Data](https://mlsanity.com/paper/2002.09599/) (2020)
- [Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization](https://mlsanity.com/paper/2110.07356/) (2021)
- [Putting GPT-3's Creativity to the (Alternative Uses) Test](https://mlsanity.com/paper/2206.08932/) (2022)
- [Sentiment Analysis of Twitter Data for Predicting Stock Market Movements](https://mlsanity.com/paper/1610.09225/) (2016)
- [Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk](https://mlsanity.com/paper/2401.05033/) (2024)
- [Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data](https://mlsanity.com/paper/2112.00054/) (2021)
- [Applying BERT and ChatGPT for Sentiment Analysis of Lyme Disease in Scientific Literature](https://mlsanity.com/paper/2302.06474/) (2023)
- [Economics Arena for Large Language Models](https://mlsanity.com/paper/2401.01735/) (2024)
- [Using Synthetic Data to Train Neural Networks is Model-Based Reasoning](https://mlsanity.com/paper/1703.00868/) (2017)
- [Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation](https://mlsanity.com/paper/2305.10361/) (2023)
- [Predicting Decisions in Language Based Persuasion Games](https://mlsanity.com/paper/2012.09966/) (2020)

## Built on by (most cited first, 12 of 12)

- [GLEE: A Unified Framework and Benchmark for Language-based Economic Environments](https://mlsanity.com/paper/2410.05254/) (2024)
- [Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets](https://mlsanity.com/paper/2510.25779/) (2025)
- [Using Reinforcement Learning to Train Large Language Models to Explain Human Decisions](https://mlsanity.com/paper/2505.11614/) (2025)
- [Can Large Language Models Replace Human Subjects? A Large-Scale Replication of Scenario-Based Experiments in Psychology and Management](https://mlsanity.com/paper/2409.00128/) (2024)
- [Alignment Makes Language Models Normative, Not Descriptive](https://mlsanity.com/paper/2603.17218/) (2026)
- [LLM Agents Display Human Biases but Exhibit Distinct Learning Patterns](https://mlsanity.com/paper/2503.10248/) (2025)
- [MALLES: A Multi-agent LLMs-based Economic Sandbox with Consumer Preference Alignment](https://mlsanity.com/paper/2603.17694/) (2026)
- [MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image](https://mlsanity.com/paper/2605.10616/) (2026)
- [CSP: A Simulator For Multi-Agent Ranking Competitions](https://mlsanity.com/paper/2502.11197/) (2025)
- [Sequential LLM Release Facilitates Manipulation in Regulated Markets](https://mlsanity.com/paper/2601.11496/) (2026)
- [RLRF: Competitive Search Agent Design via Reinforcement Learning from Ranker Feedback](https://mlsanity.com/paper/2510.04096/) (2025)
- [Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling](https://mlsanity.com/paper/2605.12411/) (2026)

## Abstract (arXiv)

Human choice prediction in economic contexts is crucial for applications in marketing, finance, public policy, and more. This task, however, is often constrained by the difficulties in acquiring human choice data. With most experimental economics studies focusing on simple choice settings, the AI community has explored whether LLMs can substitute for humans in these predictions and examined more complex experimental economics settings. However, a key question remains: can LLMs generate training data for human choice prediction? We explore this in language-based persuasion games, a complex economic setting involving natural language in strategic interactions. Our experiments show that models trained on LLM-generated data can effectively predict human behavior in these games and even outperform models trained on actual human data. Beyond data generation, we investigate the dual role of LLMs as both data generators and predictors, introducing a comprehensive empirical study on the effectiveness of utilizing LLMs for data generation, human choice prediction, or both. We then utilize our choice prediction framework to analyze how strategic factors shape decision-making, showing that interaction history (rather than linguistic sentiment alone) plays a key role in predicting human decision-making in repeated interactions. Particularly, when LLMs capture history-dependent decision patterns similarly to humans, their predictive success improves substantially. Finally, we demonstrate the robustness of our findings across alternative persuasion-game settings, highlighting the broader potential of using LLM-generated data to model human decision-making.