# Supervised structure learning

arXiv: https://arxiv.org/abs/2311.10300 · HTML: https://mlsanity.com/paper/2311.10300/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 125 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Builds on (most cited first, 22 of 22)

- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks](https://mlsanity.com/paper/1701.04722/) (2017)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Deep active inference agents using Monte-Carlo methods](https://mlsanity.com/paper/2006.04176/) (2020)
- [Sophisticated Inference](https://mlsanity.com/paper/2006.04120/) (2020)
- [Learning Disentangled Representations via Mutual Information Estimation](https://mlsanity.com/paper/1912.03915/) (2019)
- [Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons](https://mlsanity.com/paper/2006.14304/) (2020)
- [Deep Transformation-Invariant Clustering](https://mlsanity.com/paper/2006.11132/) (2020)
- [The Free Energy Principle for Perception and Action: A Deep Learning Perspective](https://mlsanity.com/paper/2207.06415/) (2022)
- [Predictive Coding, Variational Autoencoders, and Biological Connections](https://mlsanity.com/paper/2011.07464/) (2020)
- [Bayesian model reduction](https://mlsanity.com/paper/1805.07092/) (2018)
- [Towards a Theory of Evolution as Multilevel Learning](https://mlsanity.com/paper/2110.14602/) (2021)
- [Bayesian policy selection using active inference](https://mlsanity.com/paper/1904.08149/) (2019)
- [How Robust are Deep Neural Networks?](https://mlsanity.com/paper/1804.11313/) (2018)
- [Branching Time Active Inference: empirical study and complexity class analysis](https://mlsanity.com/paper/2111.11276/) (2021)
- [Active Predictive Coding Networks: A Neural Solution to the Problem of Learning Reference Frames and Part-Whole Hierarchies](https://mlsanity.com/paper/2201.08813/) (2022)
- [Neural dynamics under active inference: plausibility and efficiency of information processing](https://mlsanity.com/paper/2001.08028/) (2020)
- [Control flow in active inference systems](https://mlsanity.com/paper/2303.01514/) (2023)
- [The Free Energy Principle drives neuromorphic development](https://mlsanity.com/paper/2207.09734/) (2022)
- [Learning Generative Models for Active Inference using Tensor Networks](https://mlsanity.com/paper/2208.08713/) (2022)

## Built on by (most cited first, 9 of 9)

- [Active Inference and Intentional Behaviour](https://mlsanity.com/paper/2312.07547/) (2023)
- [Possible Principles for Aligned Structure Learning Agents](https://mlsanity.com/paper/2410.00258/) (2024)
- [Learning Dynamic Cognitive Map with Autonomous Navigation](https://mlsanity.com/paper/2411.08447/) (2024)
- [AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models](https://mlsanity.com/paper/2505.24784/) (2025)
- [Active inference and artificial reasoning](https://mlsanity.com/paper/2512.21129/) (2025)
- [Active Inference for Physical AI Agents -- An Engineering Perspective](https://mlsanity.com/paper/2603.20927/) (2026)
- [Wanting to be Understood](https://mlsanity.com/paper/2504.06611/) (2025)
- [Natural Building Blocks for Structured World Models: Theory, Evidence, and Scaling](https://mlsanity.com/paper/2511.02091/) (2025)
- [Robot Metacognition: Decision Making with Confidence for Tool Invention](https://mlsanity.com/paper/2511.16390/) (2025)

## Abstract (arXiv)

This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move - in the ensuing schemes - is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states - and their characteristic paths or dynamics.