Positive Alignment: Artificial Intelligence for Human Flourishing
Not on the map. Why?
Abstract (arXiv)
Existing alignment research is dominated by concerns about safety and preventing harm: safeguards, controllability, and compliance. This paradigm of alignment parallels early psychology's focus on mental illness: necessary but incomplete. What we call Positive Alignment is the development of AI systems that (i) actively support human and ecological flourishing in a pluralistic, polycentric, context-sensitive, and user-authored way while (ii) remaining safe and cooperative. It is a distinct and necessary agenda within AI alignment research. We argue that several existing failures of alignment (e.g., engagement hacking, loss of human autonomy, failures in truth-seeking, low epistemic humility, error correction, lack of diverse viewpoints, and being primarily reactive rather than proactive) may be better addressed through positive alignment, including cultivating virtues and maximizing human flourishing. We highlight a range of challenges, open questions, and technical directions (e.g., data filtering and upsampling, pre- and post-training, evaluations, collaborative value collection) for different phases of the LLM and agents lifecycle. We end with design principles for promoting disagreement and decentralization through contextual grounding, community customization, continual adaptation, and polycentric governance; that is, many legitimate centers of oversight rather than one institutional or moral chokepoint.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 97 of its 252 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
97 of 252 references.
References · 97
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2020Learning to summarize from human feedbackcited by 2,562
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2016Concrete Problems in AI Safetycited by 1,582
38 more
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2020Aligning AI With Shared Human Valuescited by 706
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
- 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
- 2020Artificial Intelligence, Values and Alignmentcited by 443
- 2023LaMP: When Large Language Models Meet Personalizationcited by 425
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
- 2018AI safety via debatecited by 384
- 2024The Platonic Representation Hypothesiscited by 362
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
- 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
- 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
- 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
- 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
- 2024A Roadmap to Pluralistic Alignmentcited by 175
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
- 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
- 2020Liquid Time-constant Networkscited by 160
- 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 120
- 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
- 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
- 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 94
- 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
50 most cited of 97 All 97 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Reinforcement Learning Towards Broadly and Persistently Beneficial Modelscited by 1
- 2026Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systemscited by 1
- 2026Alignment Plausibility: A New Standard for Assuring AI in Healthcarecited by 0
- 2026Closing the AI Trust Gap: The Case for Independent Certification for Trustworthy AIcited by 0
- 2026Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactionscited by 0
- 2026Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions3 Aug 2026
- 2026Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systems31 Jul 2026
- 2026Closing the AI Trust Gap: The Case for Independent Certification for Trustworthy AI17 Jul 2026
- 2026Alignment Plausibility: A New Standard for Assuring AI in Healthcare8 Jul 2026
- 2026Reinforcement Learning Towards Broadly and Persistently Beneficial Models22 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2024Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignment
- 2024Position: Towards Bidirectional Human-AI Alignment
- 2024ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
- 2026A Roadmap to Impactful Pluralistic Alignment Research
- 2026Evaluating Artificial Intelligence Through a Christian Understanding of Human Flourishing
- 2024Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignment
- 2026AI Value Alignment for Evolving Social Norms
- 2025Normative Conflicts and Shallow AI Alignment