mlsanityData to 7 Oct 2026

Research Priorities for Robust and Beneficial Artificial Intelligence

arXiv 1602.03506 ↗First posted 10 Feb 2016cs.AI stat.MLcited by 74

Not on the map. Why?

Abstract (arXiv)

Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these benefits while avoiding potential pitfalls. This article gives numerous examples (which should by no means be construed as an exhaustive list) of such worthwhile research aimed at ensuring that AI remains robust and beneficial.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 2; 2020: 0; 2021: 14; 2022: 8; 2023: 10; 2024: 9; 2025: 21; 2026: 8.

74 papers.

Sort:

By area:

  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  2. 2021Unsolved Problems in ML Safetycited by 291
  3. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  4. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  5. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  6. 2021Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AIcited by 38
  7. 2022Melting Pot 2.0cited by 37
  8. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architecturescited by 25
  9. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  10. 2023Interactive AI Alignment: Specification, Process, and Evaluation Alignmentcited by 22
  11. 2024On the Essence and Prospect: An Investigation of Alignment Approaches for Big Modelscited by 21
  12. 2019Learning Norms from Stories: A Prior for Value Aligned Agentscited by 19
38 more
  1. 2025Detecting Proxy Gaming in RL and LLM Alignment via Evaluator Stress Testscited by 19
  2. 2021Training Value-Aligned Reinforcement Learning Agents Using a Normative Priorcited by 14
  3. 2022Towards a multi-stakeholder value-based assessment framework for algorithmic systemscited by 12
  4. 2024Contestable AI needs Computational Argumentationcited by 12
  5. 2025Episodic memory in AI agents poses risks that should be studied and mitigatedcited by 11
  6. 2023Human Control: Definitions and Algorithmscited by 10
  7. 2024Auditing of AI: Legal, Ethical and Technical Approachescited by 9
  8. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  9. 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
  10. 2019Non-Cooperative Inverse Reinforcement Learningcited by 7
  11. 2021Making Human-Like Trade-offs in Constrained Environments by Learning from Demonstrationscited by 7
  12. 2021Model-Free Risk-Sensitive Reinforcement Learningcited by 7
  13. 2024Value Preferences Estimation and Disambiguation in Hybrid Participatory Systemscited by 7
  14. 2024Sociotechnical Implications of Generative Artificial Intelligence for Information Accesscited by 6
  15. 2021On managing vulnerabilities in AI/ML systemscited by 5
  16. 2021Axes for Sociotechnical Inquiry in AI Researchcited by 5
  17. 2022When Is It Acceptable to Break the Rules? Knowledge Representation of Moral Judgement Based on Empirical Datacited by 5
  18. 2023A Strong Baseline for Batch Imitation Learningcited by 5
  19. 2025Intrinsic Barriers and Practical Pathways for Human-AI Alignment: An Agreement-Based Complexity Analysiscited by 5
  20. 2026Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignmentcited by 5
  21. 2021Causal Analysis of Agent Behavior for AI Safetycited by 4
  22. 2025Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken Englishcited by 4
  23. 2025Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Makingcited by 3
  24. 2025Interpretable Risk Mitigation in LLM Agent Systemscited by 3
  25. 2021When is it permissible for artificial intelligence to lie? A trust-based approachcited by 2
  26. 2022Learning Behavioral Soft Constraints from Demonstrationscited by 2
  27. 2023CERN for AI: A Theoretical Framework for Autonomous Simulation-Based Artificial Intelligence Testing and Alignmentcited by 2
  28. 2025Information Science Principles of Machine Learning: A Causal Chain Meta-Framework Based on Formalized Information Mappingcited by 2
  29. 2025Beyond expected value: geometric mean optimization for long-term policy performance in reinforcement learningcited by 2
  30. 2022The Concept of Criticality in AI Safetycited by 1
  31. 2023AI Alignment Dialogues: An Interactive Approach to AI Alignment in Support Agentscited by 1
  32. 2023Deep Natural Language Feature Learning for Interpretable Predictioncited by 1
  33. 2024On the Undecidability of Artificial Intelligence Alignment: Machines that Haltcited by 1
  34. 2024Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignmentcited by 1
  35. 2024Can an AI Agent Safely Run a Government? Existence of Probably Approximately Aligned Policiescited by 1
  36. 2025ValuePilot: A Two-Phase Framework for Value-Driven Decision-Makingcited by 1
  37. 2025Contemplative Artificial Intelligencecited by 1
  38. 2025What Makes AI Applications Acceptable or Unacceptable? A Predictive Moral Frameworkcited by 1

50 most cited of 74 All 74 (JSON)