mlsanityData to 7 Oct 2026

An overview of 11 proposals for building safe advanced AI

arXiv 2012.07532 ↗First posted 4 Dec 2020cs.LG cs.AIcited by 18

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

This paper analyzes and compares 11 different proposals for building safe advanced AI under the current machine learning paradigm, including major contenders such as iterated amplification, AI safety via debate, and recursive reward modeling. Each proposal is evaluated on the four components of outer alignment, inner alignment, training competitiveness, and performance competitiveness, of which the distinction between the latter two is introduced in this paper. While prior literature has primarily focused on analyzing individual proposals, or primarily focused on outer alignment at the expense of inner alignment, this analysis seeks to take a comparative look at a wide range of proposals including a comparative analysis across all four previously mentioned components.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. First posted on arXiv 4 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 5; 2024: 3; 2025: 6.

18 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  3. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  4. 2023Large Language Model Alignment: A Surveycited by 241
  5. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  6. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  7. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  8. 2021Robust Feature-Level Adversaries are Interpretability Toolscited by 27
  9. 2025Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?cited by 26
  10. 2023Conditioning Predictive Models: Risks and Strategiescited by 11
  11. 2025Difficulties with Evaluating a Deception Detector for AIscited by 11
  12. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
6 more
  1. 2022Diagnostics for Deep Neural Networks with Automated Copy/Paste Attackscited by 6
  2. 2022Circumventing interpretability: How to defeat mind-readerscited by 6
  3. 2024Reinforcement Learning from LLM Feedback to Counteract Goal Misgeneralizationcited by 5
  4. 2025Discovering Transformer Circuits via a Hybrid Attribution and Pruning Frameworkcited by 4
  5. 2025Unraveling Token Prediction Refinement and Identifying Essential Layers in Language Modelscited by 3
  6. 2025Towards Integrated Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Position: AI Safety Must Embrace an Antifragile Perspective
  2. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
  3. 2024Standardization Trends on Safety and Trustworthiness Technology for Advanced AI
  4. 2026Position: AI Safety Requires Effective Controllability
  5. 2025Vulnerability Mitigation for Safety-Aligned Language Models via Debiasing
  6. 2025SafeWork-R1: Coevolving Safety and Intelligence under the AI-45circ Law
  7. 2026What Matters For Safety Alignment?
  8. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts