mlsanityData to 7 Oct 2026

Collective Constitutional AI: Aligning a Language Model with Public Input

arXiv 2406.07814 ↗First posted 12 Jun 2024cs.AI cs.CL cs.HCcited by 120

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

There is growing consensus that language model (LM) developers should not be the sole deciders of LM behavior, creating a need for methods that enable the broader public to collectively shape the behavior of LM systems that affect them. To address this need, we present Collective Constitutional AI (CCAI): a multi-stage process for sourcing and integrating public input into LMs-from identifying a target population to sourcing principles to training and evaluating a model. We demonstrate the real-world practicality of this approach by creating what is, to our knowledge, the first LM fine-tuned with collectively sourced public input and evaluating this model against a baseline model trained with established principles from a LM developer. Our quantitative evaluations demonstrate several benefits of our approach: the CCAI-trained model shows lower bias across nine social dimensions compared to the baseline model, while maintaining equivalent performance on language, math, and helpful-harmless evaluations. Qualitative comparisons of the models suggest that the models differ on the basis of their respective constitutions, e.g., when prompted with contentious topics, the CCAI-trained model tends to generate responses that reframe the matter positively instead of a refusal. These results demonstrate a promising, tractable pathway toward publicly informed development of language models.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 27 of its 114 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

27 of 114 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Case Repositories: Towards Case-Based Reasoning for AI Alignment18 Nov
  2. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
  3. 2020Learning to summarize from human feedback2 Sep

Other references · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  8. 2023Whose Opinions Do Language Models Reflect?cited by 759
  9. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  10. 2020Artificial Intelligence, Values and Alignmentcited by 443
  11. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  12. 2019Measurement and Fairnesscited by 299
12 more
  1. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  2. 2021The Values Encoded in Machine Learning Researchcited by 172
  3. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  4. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  5. 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 137
  6. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 115
  7. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  8. 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practicecited by 86
  9. 2023AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asiacited by 54
  10. 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 33
  11. 2016Learning Dynamic Robot-to-Human Object Handover from Human Feedbackcited by 19
  12. 2023Going public: the role of public participation approaches in commercial AI labscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 53; 2026: 43.

120 papers.

Sort:

By area:

  1. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 67
  2. 2025Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactionscited by 47
  3. 2024Beyond Preferences in AI Alignmentcited by 46
  4. 2025Infrastructure for AI Agentscited by 40
  5. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  6. 2024Moral Alignment for LLM Agentscited by 39
  7. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 35
  8. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 28
  9. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
  10. 2025CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMscited by 26
  11. 2024MLLM-as-a-Judge for Image Safety without Human Labelingcited by 23
  12. 2025Beyond Linear Steering: Unified Multi-Attribute Control for Language Modelscited by 23
38 more
  1. 2025AI Alignment at Your Discretioncited by 20
  2. 2024The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Modelscited by 19
  3. 2025Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamicscited by 19
  4. 2024Epistemic Injustice in Generative AIcited by 16
  5. 2024Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Surveycited by 15
  6. 2025Pre-Trained Policy Discriminators are General Reward Modelscited by 14
  7. 2025C3AI: Crafting and Evaluating Constitutions for Constitutional AIcited by 13
  8. 2024Acceptable Use Policies for Foundation Modelscited by 12
  9. 2024Selection of LLM Fine-Tuning Data based on Orthogonal Rulescited by 12
  10. 2024Beyond the Binary: Capturing Diverse Preferences With Reward Regularizationcited by 11
  11. 2025HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistantscited by 10
  12. 2025Stress-Testing Model Specs Reveals Character Differences among Language Modelscited by 10
  13. 2024ProgressGym: Alignment with a Millennium of Moral Progresscited by 9
  14. 2025Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisionscited by 9
  15. 2025Value Drifts: Tracing Value Alignment During LLM Post-Trainingcited by 9
  16. 2024Representative Social Choice: From Learning Theory to AI Alignmentcited by 8
  17. 2025Societal Alignment Frameworks Can Improve LLM Alignmentcited by 8
  18. 2025LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spacescited by 8
  19. 2024Self-Directed Synthetic Dialogues and Revisions Technical Reportcited by 7
  20. 2024Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarioscited by 7
  21. 2025The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Modelscited by 7
  22. 2025Just-In-Time Objectives: A General Approach for Specialized AI Interactionscited by 7
  23. 2025Benchmarking Overton Pluralism in LLMscited by 7
  24. 2024SPICA: Retrieving Scenarios for Pluralistic In-Context Alignmentcited by 6
  25. 2024AI and the Future of Digital Public Squarescited by 6
  26. 2024Towards Safe and Honest AI Agents with Neural Self-Other Overlapcited by 6
  27. 2025Data-adaptive Safety Rules for Training Reward Modelscited by 6
  28. 2025Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Modelscited by 6
  29. 2025Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Valuecited by 6
  30. 2024Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviorscited by 5
  31. 2024Large Language Models in Politics and Democracy: A Comprehensive Surveycited by 5
  32. 2025Decentralising LLM Alignment: A Case for Context, Pluralism, and Participationcited by 5
  33. 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
  34. 2025AI-Enhanced Deliberative Democracy and the Future of the Collective Willcited by 4
  35. 2025ENCORE: Entropy-guided Reward Composition for Multi-head Safety Reward Modelscited by 4
  36. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  37. 2024Understanding "Democratization" in NLP and ML Researchcited by 3
  38. 2024Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governancecited by 3

50 most cited of 120 All 120 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Preferences: Learning Alignment Principles Grounded in Human Reasons and Values
  2. 2025Dynamic Alignment for Collective Agency: Toward a Scalable Self-Improving Framework for Open-Ended LLM Alignment
  3. 2026Culturally-Grounded Governance for Multilingual Language Models: Rights, Data Boundaries, and Accountable AI Design
  4. 2025To Mask or to Mirror: Human-AI Alignment in Collective Reasoning
  5. 2024LLM Voting: Human Choices and AI Collective Decision Making
  6. 2026From Constitutions to Control: Interpretable Rewards for Aligning Language Models
  7. 2025How Effective Is Constitutional AI in Small LLMs? A Study on DeepSeek-R1 and Its Peers
  8. 2025Decoding Human Preferences in Alignment: An Improved Approach to Inverse Constitutional AI