RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulation
Not on the map. Why?
Abstract (arXiv)
Simulation of surveys using LLMs is emerging as a powerful application for generating human-like responses at scale. Prior work evaluates survey simulation using metrics borrowed from other domains, which are often ad hoc, fragmented, and non-standardized, leading to results that are difficult to compare. Moreover, existing metrics focus mainly on accuracy or distributional measures, overlooking the critical dimension of ranking alignment. In practice, a simulation can achieve high accuracy while still failing to capture the option most preferred by humans - a distinction that is critical in decision-making applications. We introduce RADIUS, a comprehensive two-dimensional alignment suite for survey simulation that captures: 1) RAnking alignment and 2) DIstribUtion alignment, each complemented by statistical Significance testing. RADIUS highlights the limitations of existing metrics, enables more meaningful evaluation of survey simulation, and provides an open-source implementation for reproducible and comparable assessment.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
18 of 34 references.
References · 18
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
- 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
- 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
- 2024Quantifying the Persona Effect in LLM Simulationscited by 164
- 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
- 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 64
- 2024Benchmarking Distributional Alignment of Large Language Modelscited by 58
6 more
- 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 50
- 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 48
- 2025SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviorscited by 48
- 2023User Simulation for Evaluating Information Access Systemscited by 30
- 2025Mixture-of-Personas Language Models for Population Simulationcited by 20
- 2025Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Distribution Shift Alignment Helps LLMs Simulate Survey Response Distributions
- 2026SurveyReview: A Reviewer-Aligned Benchmark for Survey Evaluators
- 2025AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys
- 2026Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data
- 2026Silicon Sampling via Cross-Survey Transfer
- 2026SurveyLens: A Discipline-Aware Benchmark for Automatic Survey Generation
- 2025SGSimEval: A Comprehensive Multifaceted and Similarity-Enhanced Benchmark for Automatic Survey Generation Systems
- 2026Can Large Language Models Revolutionize Survey Research? Experiments with Disaster Preparedness Responses