mlsanityData to 7 Oct 2026

Provably safe systems: the only path to controllable AGI

arXiv 2309.01933 ↗First posted 5 Sep 2023cs.CY cs.AI cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

We describe a path to humanity safely thriving with powerful Artificial General Intelligences (AGIs) by building them to provably satisfy human-specified requirements. We argue that this will soon be technically feasible using advanced AI for formal verification and mechanistic interpretability. We further argue that it is the only path which guarantees safe controlled AGI. We end with a list of challenge problems whose solution would contribute to this positive outcome and invite readers to join in this work.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 11 of its 81 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

11 of 81 references.

References · 11

  1. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  2. 2022Autoformalization with Large Language Modelscited by 252
  3. 2023An Overview of Catastrophic AI Riskscited by 214
  4. 2022HyperTree Proof Search for Neural Theorem Provingcited by 192
  5. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  6. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  7. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  8. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  9. 2023Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Provingcited by 37
  10. 2016The AGI Containment Problemcited by 36
  11. 2022Precision Machine Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 17; 2026: 5.

29 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  3. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  4. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  6. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  7. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 23
  8. 2024Opening the AI black box: program synthesis via mechanistic interpretabilitycited by 19
  9. 2024Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selectioncited by 13
  10. 2025Distributional AGI Safetycited by 8
  11. 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debatescited by 7
  12. 2025AI Testing Should Account for Sophisticated Strategic Behaviourcited by 6
17 more
  1. 2024Investigating Representation Universality: Case Study on Genealogical Representationscited by 4
  2. 2025On the creation of narrow AI: hierarchy and nonlocality of neural network skillscited by 4
  3. 2025Free Energy Risk Metrics for Systemically Safe AI: Gatekeeping Multi-Agent Studycited by 2
  4. 2025The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AIcited by 2
  5. 2025Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecuritycited by 1
  6. 2025No-Knowledge Alarms for Misaligned LLMs-as-Judgescited by 1
  7. 2025Bounded PCTL Model Checking of Large Language Model Outputscited by 1
  8. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  9. 2025Verifying Memoryless Sequential Decision-making of Large Language Modelscited by 1
  10. 2026Owner-Harm: A Missing Threat Model for AI Agent Safetycited by 1
  11. 2025Towards provable probabilistic safety for scalable embodied AI systemscited by 0
  12. 2025Limitations on Accurate, Trusted, Human-level Reasoningcited by 0
  13. 2025Logical Consistency Between Disagreeing Experts and Its Role in AI Safetycited by 0
  14. 2026Provably Secure Agent Guardrailcited by 0
  15. 2026International Agreements to Limit Frontier AI: Objectives and Exitcited by 0
  16. 2026Contract monitoring: governing AI via separation of powerscited by 0
  17. 2026AI Safety Considerations for Agents With Limited Time to Actcited by 0