mlsanityData to 7 Oct 2026

Hard Choices in Artificial Intelligence

arXiv 2106.11022 ↗First posted 10 Jun 2021cs.CY cs.AI cs.SYcited by 26

Not on the map. Why?

Abstract (arXiv)

As AI systems are integrated into high stakes social domains, researchers now examine how to design and operate them in a safe and ethical manner. However, the criteria for identifying and diagnosing safety risks in complex social contexts remain unclear and contested. In this paper, we examine the vagueness in debates about the safety and ethical behavior of AI systems. We show how this vagueness cannot be resolved through mathematical formalism alone, instead requiring deliberation about the politics of development as well as the context of deployment. Drawing from a new sociotechnical lexicon, we redefine vagueness in terms of distinct design challenges at key stages in AI system development. The resulting framework of Hard Choices in Artificial Intelligence (HCAI) empowers developers by 1) identifying points of overlap between design decisions and major sociotechnical challenges; 2) motivating the creation of stakeholder feedback channels so that safety issues can be exhaustively addressed. As such, HCAI contributes to a timely debate about the status of AI development in democratic societies, arguing that deliberation should be the goal of AI Safety, not just the procedure by which it is ensured.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 17 of its 140 references that are in the map. First posted on arXiv 10 Jun 2021.

Builds on

17 of 140 references.

References · 17

  1. 2018Datasheets for Datasetscited by 1,371
  2. 2018Model Cards for Model Reportingcited by 1,068
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2019On Evaluating Adversarial Robustnesscited by 442
  5. 2016Cooperative Inverse Reinforcement Learningcited by 298
  6. 2017Inverse Reward Designcited by 246
  7. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  8. 2017Fairness in Machine Learning: Lessons from Political Philosophycited by 148
  9. 2019Principles alone cannot guarantee ethical AIcited by 134
  10. 2018Incomplete Contracting and AI Alignmentcited by 60
  11. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  12. 2018Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)cited by 22
5 more
  1. 2019On the Utility of Model Learning in HRIcited by 21
  2. 2017Should Robots be Obedient?cited by 20
  3. 2018A Broader View on Bias in Automated Decision-Making: Reflecting on Epistemology and Dynamicscited by 10
  4. 2020Regulating Artificial Intelligence: Proposal for a Global Solutioncited by 10
  5. 2020Privacy Engineering Meets Software Engineering. On the Challenges of Engineering Privacy ByDesigncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 8; 2024: 3; 2025: 6; 2026: 4.

26 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2022Defining and Characterizing Reward Hackingcited by 184
  3. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  4. 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedbackcited by 95
  5. 2023The History and Risks of Reinforcement Learning and Human Feedbackcited by 42
  6. 2024AI Alignment with Changing and Influenceable Reward Functionscited by 33
  7. 2022System Safety and Artificial Intelligencecited by 18
  8. 2025Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effectscited by 18
  9. 2022Tiny, always-on and fragile: Bias propagation through design choices in on-device machine learning workflowscited by 8
  10. 2023ACROCPoLis: A Descriptive Framework for Making Sense of Fairnesscited by 8
  11. 2025Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articlescited by 6
  12. 2023Regulation and NLP (RegNLP): Taming Large Language Modelscited by 5
14 more
  1. 2023Optimization's Neglected Normative Commitmentscited by 4
  2. 2022Closed-Loop View of the Regulation of AI: Equal Impact across Repeated Interactionscited by 3
  3. 2023The Value-Sensitive Conversational Agent Co-Design Frameworkcited by 3
  4. 2024Developing and Evaluating a Design Method for Positive Artificial Intelligencecited by 2
  5. 2025Explainable AI Systems Must Be Contestable: Here's How to Make It Happencited by 2
  6. 2023Positive AI: Key Challenges in Designing Artificial Intelligence for Wellbeingcited by 1
  7. 2026CIRCLE: A Framework for Evaluating AI from a Real-World Lenscited by 1
  8. 2021Demanding and Designing Aligned Cognitive Architecturescited by 0
  9. 2025From Data-Driven to Purpose-Driven Artificial Intelligence: Systems Thinking for Data-Analytic Automation of Patient Carecited by 0
  10. 2025Normative Moral Pluralism for AI: A Framework for Deliberation in Complex Moral Contextscited by 0
  11. 2025From Mimicry to True Intelligence (TI) -- A New Paradigm for Artificial General Intelligencecited by 0
  12. 2026RCTs for Frontier AI Governance: Methodological Challenges and Solutions for Human Uplift Studiescited by 0
  13. 2026Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemmacited by 0
  14. 2026Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignmentcited by 0