# GPS-Bench: A Governance Policy Benchmark for Automating Policy Analysis

arXiv: https://arxiv.org/abs/2609.03553 · HTML: https://mlsanity.com/paper/2609.03553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 3 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents](https://mlsanity.com/paper/2404.16698/) (2024)
- [LLM Social Simulations Are a Promising Research Method](https://mlsanity.com/paper/2504.02234/) (2025)
- [Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database](https://mlsanity.com/paper/2011.08512/) (2020)
- [Escalation Risks from Language Models in Military and Diplomatic Decision-Making](https://mlsanity.com/paper/2401.03408/) (2024)
- [Thousands of AI Authors on the Future of AI](https://mlsanity.com/paper/2401.02843/) (2024)
- [Computing Power and the Governance of Artificial Intelligence](https://mlsanity.com/paper/2402.08797/) (2024)
- [Simulating Policy Impacts: Developing a Generative Scenario Writing Method to Evaluate the Perceived Effects of Regulation](https://mlsanity.com/paper/2405.09679/) (2024)
- [NomicLaw: Emergent Trust and Strategic Argumentation in LLMs During Collaborative Law-Making](https://mlsanity.com/paper/2508.05344/) (2025)
- [AI, Digital Platforms, and the New Systemic Risk](https://mlsanity.com/paper/2509.17878/) (2025)
- [The Nuclear Analogy in AI Governance Research](https://mlsanity.com/paper/2510.21203/) (2025)

## Abstract (arXiv)

Policy analysis requires more than predicting whether a proposal will pass: it requires identifying who will be affected, how those actors respond, and what follows. LLM-based policy simulations model these processes at scale, but their validity is hard to establish when plausible behaviour is never compared with observed outcomes. We introduce GPS-Bench, an evidence-grounded benchmark for governance policy simulation that links policies to relevant actors, actor actions and downstream impacts using legislative records, lobbying disclosures, regulatory documents, corporate filings, economic data and other public evidence. Actors are reconstructed from the dated record rather than prompted as archetypes, so a persona is an evidence object with provenance; a human-annotated pool forms the Gold evaluation set, while cases labelled by a separate LLM from retrieved evidence are treated as Silver supervision and never as test labels. Because every inference mode reads the same grounded state and emits the same schema, GPS-Bench turns "does multi-agent simulation help?" into a controlled comparison: we contrast joint reasoning, independent and communicating actor agents, graph-based methods and weight-level fine-tuning over one policy state. Fine-tuning on the grounded record gives the strongest actor-level impact prediction, and decomposition does not beat it; what decomposition adds is mechanism. Agents hold private, non-identical evidence, each seeing its own exposure clause, and address named partners with concrete joint proposals, what they offer, what they need in return, and why acting together beats acting alone, so the coalitions that form can be checked against the commitments the record holds. GPS-Bench therefore gives a common empirical setting for studying when evidence, actor modelling and multi-agent interaction improve the prediction and interpretation of policy outcomes.