# Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy

arXiv: https://arxiv.org/abs/2503.09639 · HTML: https://mlsanity.com/paper/2503.09639/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 74 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 26 of 26)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents](https://mlsanity.com/paper/2405.02957/) (2024)
- [AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society](https://mlsanity.com/paper/2502.08691/) (2025)
- [EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities](https://mlsanity.com/paper/2310.10436/) (2023)
- [Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game](https://mlsanity.com/paper/2310.18940/) (2023)
- [Simulating Classroom Education with LLM-Empowered Agents](https://mlsanity.com/paper/2406.19226/) (2024)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs](https://mlsanity.com/paper/2403.05020/) (2024)
- [Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions](https://mlsanity.com/paper/2410.02584/) (2024)
- [GenSim: A General Social Simulation Platform with Large Language Model based Agents](https://mlsanity.com/paper/2410.04360/) (2024)
- [Epidemic Modeling with Generative Agents](https://mlsanity.com/paper/2307.04986/) (2023)
- [Emergence of Social Norms in Generative Agent Societies: Principles and Architecture](https://mlsanity.com/paper/2403.08251/) (2024)
- [AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents](https://mlsanity.com/paper/2408.08089/) (2024)
- [LLMs generate structurally realistic social networks but overestimate political homophily](https://mlsanity.com/paper/2408.16629/) (2024)
- [On the limits of agency in agent-based models](https://mlsanity.com/paper/2409.10568/) (2024)
- [A Simulation System Towards Solving Societal-Scale Manipulation](https://mlsanity.com/paper/2410.13915/) (2024)
- [Accurate Measures of Vaccination and Concerns of Vaccine Holdouts from Web Search Logs](https://mlsanity.com/paper/2306.07457/) (2023)
- [Deep Learning Reveals Patterns of Diverse and Changing Sentiments Towards COVID-19 Vaccines Based on 11 Million Tweets](https://mlsanity.com/paper/2207.10641/) (2022)

## Built on by (most cited first, 17 of 17)

- [Social World Models](https://mlsanity.com/paper/2509.00559/) (2025)
- [The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies](https://mlsanity.com/paper/2509.18052/) (2025)
- [Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research](https://mlsanity.com/paper/2506.01839/) (2025)
- [Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits](https://mlsanity.com/paper/2605.18890/) (2026)
- [Do Role-Playing Agents Practice What They Preach? Belief-Behavior Consistency in LLM-Based Simulations of Human Trust](https://mlsanity.com/paper/2507.02197/) (2025)
- [Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values](https://mlsanity.com/paper/2506.00079/) (2025)
- [Improving the Distributional Alignment of LLMs using Supervision](https://mlsanity.com/paper/2507.00439/) (2025)
- [What Makes LLM Agent Simulations Useful for Policy Practice? An Iterative Design Study in Emergency Preparedness](https://mlsanity.com/paper/2509.21868/) (2025)
- [SimCity: Multi-Agent Urban Development Simulation with Rich Interactions](https://mlsanity.com/paper/2510.01297/) (2025)
- [Sentipolis: Emotion-Aware Agents for Social Simulations](https://mlsanity.com/paper/2601.18027/) (2026)
- [AI-VaxGuide: An Agentic RAG-Based LLM for Vaccination Decisions](https://mlsanity.com/paper/2507.03493/) (2025)
- [Step-Level Preference Learning for Generative Agents in Social Simulations](https://mlsanity.com/paper/2607.14485/) (2026)
- [Informing AI Policy Assessment using Large-Scale Simulation of Interventions](https://mlsanity.com/paper/2605.27395/) (2026)
- [SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game](https://mlsanity.com/paper/2606.27397/) (2026)
- [Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions](https://mlsanity.com/paper/2609.38555/) (2026)
- [EpiWorld: Grounding LLM Policy Agents in Epidemiological World Models](https://mlsanity.com/paper/2610.02744/) (2026)
- [PsyBer-Agent: A Psychology-Driven User Agent for Recommender Systems](https://mlsanity.com/paper/2610.13989/) (2026)

## Abstract (arXiv)

Can we simulate a sandbox society with generative agents to model human behavior, thereby reducing the over-reliance on real human trials for assessing public policies? In this work, we investigate the feasibility of simulating health-related decision-making, using vaccine hesitancy, defined as the delay in acceptance or refusal of vaccines despite the availability of vaccination services (MacDonald, 2015), as a case study. To this end, we introduce the VacSim framework with 100 generative agents powered by Large Language Models (LLMs). VacSim simulates vaccine policy outcomes with the following steps: 1) instantiate a population of agents with demographics based on census data; 2) connect the agents via a social network and model vaccine attitudes as a function of social dynamics and disease-related information; 3) design and evaluate various public health interventions aimed at mitigating vaccine hesitancy. To align with real-world results, we also introduce simulation warmup and attitude modulation to adjust agents' attitudes. We propose a series of evaluations to assess the reliability of various LLM simulations. Experiments indicate that models like Llama and Qwen can simulate aspects of human behavior but also highlight real-world alignment challenges, such as inconsistent responses with demographic profiles. This early exploration of LLM-driven simulations is not meant to serve as definitive policy guidance; instead, it serves as a call for action to examine social simulation for policy development.