mlsanityData to 14 Oct 2026

Resource Rational Contractualism Should Guide AI Alignment

arXiv 2506.17434 ↗First posted 20 Jun 2025cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

AI systems will soon have to navigate human environments and make decisions that affect people and other AI agents whose goals and values diverge. Contractualist alignment proposes grounding those decisions in agreements that diverse stakeholders would endorse under the right conditions, yet securing such agreement at scale remains costly and slow -- even for advanced AI. We therefore propose Resource-Rational Contractualism (RRC): a framework where AI systems approximate the agreements rational parties would form by drawing on a toolbox of normatively-grounded, cognitively-inspired heuristics that trade effort for accuracy. An RRC-aligned agent would not only operate efficiently, but also be equipped to dynamically adapt to and interpret the ever-changing human social world.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 31 of its 95 references that are in the map. First posted on arXiv 20 Jun 2025.

Builds on

31 of 95 references.

References · 31

  1. 2023Let's Verify Step by Stepcited by 3,789
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2020Artificial Intelligence, Values and Alignmentcited by 448
  4. 2018AI safety via debatecited by 400
  5. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  6. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  7. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 301
  8. 2023AI Alignment: A Comprehensive Surveycited by 263
  9. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  10. 2023Large Language Model Alignment: A Surveycited by 243
  11. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 119
  12. 2024How Well Can LLMs Negotiate? NegotiationArena Platform and Analysiscited by 100
19 more
  1. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 97
  2. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 96
  3. 2018Incomplete Contracting and AI Alignmentcited by 62
  4. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  5. 2024Beyond Preferences in AI Alignmentcited by 49
  6. 2023Testing Language Model Agents Safely in the Wildcited by 46
  7. 2025Infrastructure for AI Agentscited by 41
  8. 2024What are human values, and how do we align AI to them?cited by 32
  9. 2025Value Profiles for Encoding Human Variationcited by 29
  10. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  11. 2023The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputscited by 23
  12. 2024Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Modelscited by 23
  13. 2013Reasoning About Beliefs and Actions Under Computational Resource Constraintscited by 18
  14. 2024Human Simulacra: Benchmarking the Personification of Large Language Modelscited by 17
  15. 2024Goals as Reward-Producing Programscited by 14
  16. 2024Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Gamescited by 11
  17. 2024AI and the Future of Digital Public Squarescited by 6
  18. 2025AGI, Governments, and Free Societiescited by 2
  19. 2024Intuitions of Compromise: Utilitarianism vs. Contractualismcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Emergent alignment and the projectability of ethical personascited by 1
  2. 2026Constructive Alignment: Governing Preference Dynamics in Human-AI Interactioncited by 1
  3. 2026Unsupervised Elicitation of Moral Values from Language Modelscited by 0
  4. 2026The Design and Composition of Structural Causal Decision Processescited by 0
  5. 2026Bounded Morality: Defining the Space of Moral Computationcited by 0
  6. 2026A Contractualist Argumentation Framework for Moral Decision-Makingcited by 0
  7. 2026Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignmentcited by 0
  8. 2026Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?cited by 0