# What Would an LLM Do? Evaluating Large Language Models for Policymaking to Alleviate Homelessness

arXiv: https://arxiv.org/abs/2509.03827 · HTML: https://mlsanity.com/paper/2509.03827/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [Talking About Large Language Models](https://mlsanity.com/paper/2212.03551/) (2022)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://mlsanity.com/paper/2412.17259/) (2024)
- [The illusion of artificial inclusion](https://mlsanity.com/paper/2401.08572/) (2024)
- [A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives](https://mlsanity.com/paper/2503.13415/) (2025)
- [Agent-Based Modelling Meets Generative AI in Social Network Simulations](https://mlsanity.com/paper/2411.16031/) (2024)
- [AgentMove: A Large Language Model based Agentic Framework for Zero-shot Next Location Prediction](https://mlsanity.com/paper/2408.13986/) (2024)
- [On the limits of agency in agent-based models](https://mlsanity.com/paper/2409.10568/) (2024)
- [OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants](https://mlsanity.com/paper/2406.14883/) (2024)
- [Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management](https://mlsanity.com/paper/2502.10407/) (2025)
- [Can Poverty Be Reduced by Acting on Discrimination? An Agent-based Model for Policy Making](https://mlsanity.com/paper/2403.01600/) (2024)
- [Value-Enriched Population Synthesis: Integrating a Motivational Layer](https://mlsanity.com/paper/2408.09407/) (2024)

## Built on by (most cited first, 1 of 1)

- [The Fragility of Strategic Thinking in Large Language Models](https://mlsanity.com/paper/2510.10813/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly being adopted in high-stakes domains. Their potential to encode evolving social contexts and to generate plausible scenarios position them as promising tools in social policymaking. This article evaluates whether LLMs are aligned with domain experts (and among themselves) on policy recommendations to alleviate homelessness - a challenge affecting over 150 million people worldwide. We develop a novel benchmark comprised of decision scenarios across four cities, with policy choices that are grounded in the conceptual framework of the Capability Approach for human development. We also present an automated pipeline that connects the policies to an agent-based model in one location, and compare the social impact of the policies recommended by LLMs to those recommended by experts. Our exploratory analysis reveals variation across LLMs in their policy recommendations compared to local experts, yet suggests potential benefits of the use of LLMs to provide insights for policymaking, if paired with responsible guardrails, contextual calibrations, and local domain expertise. Our work operationalizes the Capability Approach in a computational framework and provides new insights on homelessness alleviation policymaking with a focus on human dignity.