mlsanityData to 7 Oct 2026

AI Alignment: A Comprehensive Survey

arXiv 2310.19852 ↗First posted 30 Oct 2023cs.AIcited by 256

Not on the map. Why?

Cited by 88 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

AI alignment aims to make AI systems behave in line with human intentions and values. As AI systems grow more capable, so do risks from misalignment. To provide a comprehensive and up-to-date overview of the alignment field, in this survey, we delve into the core concepts, methodology, and practice of alignment. First, we identify four principles as the key objectives of AI alignment: Robustness, Interpretability, Controllability, and Ethicality (RICE). Guided by these four principles, we outline the landscape of current alignment research and decompose them into two key components: forward alignment and backward alignment. The former aims to make AI systems aligned via alignment training, while the latter aims to gain evidence about the systems' alignment and govern them appropriately to avoid exacerbating misalignment risks. On forward alignment, we discuss techniques for learning from feedback and learning under distribution shift. On backward alignment, we discuss assurance techniques and governance practices. We also release and continually update the website (www.alignmentsurvey.com) which features tutorials, collections of papers, blog posts, and other resources.

Key facts. Cited by 256 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Oct 2023.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 103; 2025: 111; 2026: 36.

256 papers.

Sort:

By area:

  1. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  2. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  3. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  4. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  5. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 252
  6. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  7. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  8. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  9. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  10. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  11. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  12. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
38 more
  1. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  2. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  3. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  4. 2024Panacea: Pareto Alignment via Preference Adaptation for LLMscited by 78
  5. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  6. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  7. 2024Can Large Language Model Agents Simulate Human Trust Behavior?cited by 62
  8. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  9. 2024PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferencescited by 53
  10. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  11. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  12. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  13. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 48
  14. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 46
  15. 2024A Language Model's Guide Through Latent Spacecited by 43
  16. 2024TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weightscited by 42
  17. 2024Impact of Preference Noise on the Alignment Performance of Generative Language Modelscited by 41
  18. 2024Quantifying the Gain in Weak-to-Strong Generalizationcited by 40
  19. 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
  20. 2024Moral Alignment for LLM Agentscited by 39
  21. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 38
  22. 2025SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safetycited by 37
  23. 2024Personality Alignment of Large Language Modelscited by 36
  24. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
  25. 2025A Survey of Direct Preference Optimizationcited by 35
  26. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  27. 2024How Susceptible are LLMs to Influence in Prompts?cited by 32
  28. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  29. 2024Stepwise Alignment for Constrained Language Model Policy Optimizationcited by 30
  30. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  31. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
  32. 2024Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Expertscited by 28
  33. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  34. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
  35. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  36. 2024Strong and weak alignment of large language models with human valuescited by 27
  37. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  38. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 27

50 most cited of 256 All 256 (JSON)