mlsanityData to 14 Oct 2026

Math for AI safety: an invitation for mathematicians

arXiv 2609.15289 ↗First posted 14 Sep 2026math.HO cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence threatens to outrun human understanding and control. New mathematics is needed to design AI that is legible, steerable, and cooperative with humanity. I organize this invitation by mathematical field, so you can turn straight to your own: logic and game theory for cooperation; probability for agency and world-models; algebra and representation theory for learned features; analysis and geometry for generalization and training dynamics. Each section ends with an open problem that is accessible to a working mathematician with no prior experience in AI safety.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 92 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

38 of 92 references.

References · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  7. 2022Toy Models of Superpositioncited by 971
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  9. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  11. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  12. 2016Cooperative Inverse Reinforcement Learningcited by 300
26 more
  1. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  2. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  3. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  4. 2025Measuring AI Ability to Complete Long Software Taskscited by 156
  5. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 144
  6. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 143
  7. 2019The Lean mathematical librarycited by 134
  8. 2021The Bayesian Learning Rulecited by 99
  9. 2024Robust agents learn causal world modelscited by 81
  10. 2017Concentration of tempered posteriors and of their variational approximationscited by 66
  11. 2023The Local Learning Coefficient: A Singularity-Aware Complexity Measurecited by 51
  12. 2025Gradual Disempowerment: Systemic Existential Risks from Incremental AI Developmentcited by 46
  13. 2017Occam's razor is insufficient to infer the preferences of irrational agentscited by 45
  14. 2020Whence the Expected Free Energy?cited by 40
  15. 2025Open Character Training: Shaping the Persona of AI Assistants through Constitutional AIcited by 24
  16. 2023Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networkscited by 23
  17. 2025You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisationcited by 17
  18. 2016Logical Inductioncited by 16
  19. 2014Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logiccited by 13
  20. 2026Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Modelscited by 12
  21. 2014Sparse and spurious: dictionary learning with noise and outlierscited by 11
  22. 2022Cooperative and uncooperative institution designs: Surprises and problems in open-source game theorycited by 11
  23. 2026Progress in Formalizing Sphere Packing in Dimension 8cited by 9
  24. 2013Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-rank Matricescited by 8
  25. 2024Characterising Simulation-Based Program Equilibriacited by 3
  26. 2025Natural Latents: Latent Variables Stable Across Ontologiescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Human agency in initial human-AI proof formalization workflows
  2. 2024Mathematics and Machine Creativity: A Survey on Bridging Mathematics with AI
  3. 2026AI co-mathematician: Accelerating mathematicians with agentic AI
  4. 2025AI Mathematician: Towards Fully Automated Frontier Mathematical Research
  5. 2025AI Mathematician as a Partner in Advancing Mathematical Discovery -- A Case Study in Homogenization Theory
  6. 2023Safety without alignment
  7. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
  8. 2024Machine learning and information theory concepts towards an AI Mathematician