mlsanityData to 7 Oct 2026

Positive Alignment: Artificial Intelligence for Human Flourishing

arXiv 2605.10310 ↗First posted 11 May 2026cs.AI cs.CY cs.HCcited by 5

Not on the map. Why?

Abstract (arXiv)

Existing alignment research is dominated by concerns about safety and preventing harm: safeguards, controllability, and compliance. This paradigm of alignment parallels early psychology's focus on mental illness: necessary but incomplete. What we call Positive Alignment is the development of AI systems that (i) actively support human and ecological flourishing in a pluralistic, polycentric, context-sensitive, and user-authored way while (ii) remaining safe and cooperative. It is a distinct and necessary agenda within AI alignment research. We argue that several existing failures of alignment (e.g., engagement hacking, loss of human autonomy, failures in truth-seeking, low epistemic humility, error correction, lack of diverse viewpoints, and being primarily reactive rather than proactive) may be better addressed through positive alignment, including cultivating virtues and maximizing human flourishing. We highlight a range of challenges, open questions, and technical directions (e.g., data filtering and upsampling, pre- and post-training, evaluations, collaborative value collection) for different phases of the LLM and agents lifecycle. We end with design principles for promoting disagreement and decentralization through contextual grounding, community customization, continual adaptation, and polycentric governance; that is, many legitimate centers of oversight rather than one institutional or moral chokepoint.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 97 of its 252 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

97 of 252 references.

References · 97

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  12. 2016Concrete Problems in AI Safetycited by 1,582
38 more
  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  3. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  8. 2022Red Teaming Language Models with Language Modelscited by 983
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  11. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  12. 2023Whose Opinions Do Language Models Reflect?cited by 759
  13. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  14. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  15. 2020Aligning AI With Shared Human Valuescited by 706
  16. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  17. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  18. 2020Artificial Intelligence, Values and Alignmentcited by 443
  19. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  20. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  21. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  22. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  23. 2018AI safety via debatecited by 384
  24. 2024The Platonic Representation Hypothesiscited by 362
  25. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  26. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
  27. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  28. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  29. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  30. 2024A Roadmap to Pluralistic Alignmentcited by 175
  31. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  32. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  33. 2020Liquid Time-constant Networkscited by 160
  34. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 120
  35. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
  36. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
  37. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 94
  38. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89

50 most cited of 97 All 97 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignment
  2. 2024Position: Towards Bidirectional Human-AI Alignment
  3. 2024ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
  4. 2026A Roadmap to Impactful Pluralistic Alignment Research
  5. 2026Evaluating Artificial Intelligence Through a Christian Understanding of Human Flourishing
  6. 2024Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignment
  7. 2026AI Value Alignment for Evolving Social Norms
  8. 2025Normative Conflicts and Shallow AI Alignment