mlsanityData to 7 Oct 2026

Possible Principles for Aligned Structure Learning Agents

arXiv 2410.00258 ↗First posted 30 Sep 2024cs.AI q-bio.NCcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper offers a roadmap for the development of scalable aligned artificial intelligence (AI) from first principle descriptions of natural intelligence. In brief, a possible path toward scalable aligned AI rests upon enabling artificial agents to learn a good model of the world that includes a good model of our preferences. For this, the main objective is creating agents that learn to represent the world and other agents' world models; a problem that falls under structure learning (a.k.a. causal representation learning or model discovery). We expose the structure learning and alignment problems with this goal in mind, as well as principles to guide us forward, synthesizing various ideas across mathematics, statistics, and cognitive science. 1) We discuss the essential role of core knowledge, information geometry and model reduction in structure learning, and suggest core structural modules to learn a wide range of naturalistic worlds. 2) We outline a way toward aligned agents through structure learning and theory of mind. As an illustrative example, we mathematically sketch Asimov's Laws of Robotics, which prescribe agents to act cautiously to minimize the ill-being of other agents. We supplement this example by proposing refined approaches to alignment. These observations may guide the development of artificial intelligence in helping to scale existing -- or design new -- aligned structure learning systems.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 50 of its 143 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

50 of 143 references.

References · 50

  1. 2016Variational Inference: A Review for Statisticianscited by 1,466
  2. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
  3. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  4. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  5. 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 329
  6. 2022On Neural Differential Equationscited by 264
  7. 2021Curriculum Learning: A Surveycited by 257
  8. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  9. 2021DiBS: Differentiable Bayesian Structure Learningcited by 105
  10. 2022GFlowNets and variational inferencecited by 103
  11. 2020Active inference on discrete state-spaces: a synthesiscited by 91
  12. 2022Amortized Inference for Causal Structure Learningcited by 89
38 more
  1. 2013Loopy Belief Propagation for Approximate Inference: An Empirical Studycited by 76
  2. 2017The free energy principle for action and perception: A mathematical reviewcited by 66
  3. 2020Sophisticated Inferencecited by 56
  4. 2021Active Inference in Robotics and Artificial Agents: Survey and Challengescited by 52
  5. 2021Agent Incentives: A Causal Perspectivecited by 51
  6. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
  7. 2021Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planningcited by 43
  8. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  9. 2018Penalizing side effects using stepwise relative reachabilitycited by 38
  10. 2015Partition MCMC for inference on acyclic digraphscited by 33
  11. 2020Action and Perception as Divergence Minimizationcited by 31
  12. 2022Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayescited by 30
  13. 2022The free energy principle made simpler but not too simplecited by 27
  14. 2022Hybrid Predictive Coding: Inferring, Fast and Slowcited by 25
  15. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  16. 2024The Reasons that Agents Act: Intention and Instrumental Goalscited by 23
  17. 2022Path integrals, particular kinds, and strange thingscited by 20
  18. 2013The Recovery of Causal Poly-Trees from Statistical Datacited by 19
  19. 2018Bayesian model reductioncited by 19
  20. 2024Three Dogmas of Reinforcement Learningcited by 14
  21. 2014Variational Particle Approximationscited by 13
  22. 2024From pixels to planning: scale-free active inferencecited by 13
  23. 2022Recognising the importance of preference change: A call for a coordinated multidisciplinary research effort in the age of AIcited by 12
  24. 2024Active Inference as a Model of Agencycited by 12
  25. 2021Bayesian Mechanics for Stationary Processescited by 10
  26. 2023Collective behavior from surprise minimizationcited by 10
  27. 2023Supervised structure learningcited by 9
  28. 2021Active inference, Bayesian optimal design, and expected utilitycited by 6
  29. 2020Neural dynamics under active inference: plausibility and efficiency of information processingcited by 5
  30. 2024Exploring and Learning Structure: Active Inference Approach in Navigational Agentscited by 5
  31. 2020Broken detailed balance and entropy production in the human braincited by 4
  32. 2023Active Inference and Intentional Behaviourcited by 4
  33. 2024Value of Information and Reward Specification in Active Inference and POMDPscited by 4
  34. 2023Towards a Bayesian mechanics of metacognitive particles: A commentary on "Path integrals, particular kinds, and strange things" by Friston, Da Costa, Sakthivadivel, Heins, Pavliotis, Ramstead, and Parrcited by 2
  35. 2016The Computational Power of Dynamic Bayesian Networkscited by 1
  36. 2024Metacognitive particles, mental action and the sense of agencycited by 1
  37. 2024A theory of generalised coordinates for stochastic differential equationscited by 1
  38. 2024Toward Universal and Interpretable World Models for Open-ended Learning Agentscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Active inference and artificial reasoningcited by 3
  2. 2025Teleology-Driven Affective Computing: A Causal Framework for Sustained Well-Beingcited by 0
  3. 2025Natural Building Blocks for Structured World Models: Theory, Evidence, and Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning
  2. 2024On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models
  3. 2025Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks
  4. 2026SMGI: A Structural Theory of General Artificial Intelligence
  5. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems
  6. 2022Designing Ecosystems of Intelligence from First Principles
  7. 2025Emergent Cognitive Convergence via Implementation: Structured Cognitive Loop Reflecting Four Theories of Mind
  8. 2025Intrinsic Barriers and Practical Pathways for Human-AI Alignment: An Agreement-Based Complexity Analysis