mlsanityData to 7 Oct 2026

A Statistical Case Against Empirical Human-AI Alignment

arXiv 2502.14581 ↗First posted 20 Feb 2025cs.AI cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Empirical human-AI alignment aims to make AI systems act in line with observed human behavior. While noble in its goals, we argue that empirical alignment can inadvertently introduce statistical biases that warrant caution. This position paper thus advocates against naive empirical alignment, offering prescriptive alignment and a posteriori empirical alignment as alternatives. We substantiate our principled argument by tangible examples like human-centric decoding of language models.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 71 of its 151 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

71 of 151 references.

References · 71

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2019Advances and Open Problems in Federated Learningcited by 3,021
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
38 more
  1. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  2. 2017Continual Learning with Deep Generative Replaycited by 1,152
  3. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  4. 2020Aligning AI With Shared Human Valuescited by 706
  5. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  6. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  7. 2020Artificial Intelligence, Values and Alignmentcited by 443
  8. 2019How to Fine-Tune BERT for Text Classification?cited by 409
  9. 2020Performative Predictioncited by 318
  10. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  11. 2023AI Alignment: A Comprehensive Surveycited by 256
  12. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  13. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  14. 2020Explainable Reinforcement Learning: A Surveycited by 139
  15. 2018Preference-based Online Learning with Dueling Bandits: A Surveycited by 100
  16. 2021Outside the Echo Chamber: Optimizing the Performative Riskcited by 95
  17. 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedbackcited by 95
  18. 2020Stochastic Optimization for Performative Predictioncited by 92
  19. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 69
  20. 2014Optimal Inference After Model Selectioncited by 63
  21. 2022Contrastive Search Is What You Need For Neural Text Generationcited by 58
  22. 2023Performative Prediction: Past and Futurecited by 52
  23. 2018A Simple and Effective Model-Based Variable Importance Measurecited by 50
  24. 2014Exact Post-Selection Inference for Sequential Regression Procedurescited by 47
  25. 2018Establishing Appropriate Trust via Critical Statescited by 47
  26. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
  27. 2024Inverse Constitutional AI: Compressing Preferences into Principlescited by 39
  28. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 38
  29. 2019Generation of Policy-Level Explanations for Reinforcement Learningcited by 37
  30. 2024AI Alignment with Changing and Influenceable Reward Functionscited by 33
  31. 2022A Survey of Explainable Reinforcement Learningcited by 31
  32. 2024Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generationcited by 27
  33. 2024Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarkscited by 26
  34. 2024Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generationcited by 26
  35. 2021Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Processcited by 24
  36. 2022Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomenacited by 21
  37. 2022Statistical Comparisons of Classifiers by Generalized Stochastic Dominancecited by 20
  38. 2024Optimal Design for Human Preference Elicitationcited by 17

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025LLM Probability Concentration: How Alignment Shrinks the Generative Horizoncited by 26
  2. 2025Statistical Multicriteria Evaluation of LLM-Generated Textcited by 9
  3. 2026The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomescited by 3
  4. 2025Human Preferences for Constructive Interactions in Language Model Alignmentcited by 1
  5. 2025The Algebra of Meaning: Why Machines Need Montague More Than Moore's Lawcited by 0
  6. 2025Empirical Decision Theorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Position: Towards Bidirectional Human-AI Alignment
  2. 2026Are Human-Aligned Models Models of Humans? A Turing-Test Gap in Preference Alignment
  3. 2024The AI Alignment Paradox
  4. 2024Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization
  5. 2024Strong and weak alignment of large language models with human values
  6. 2026Toward a Theory of Value in AI Alignment
  7. 2024Challenges and Future Directions of Data-Centric AI Alignment
  8. 2023The Alignment Problem in Context