# Inferring Inference

arXiv: https://arxiv.org/abs/2310.03186 · HTML: https://mlsanity.com/paper/2310.03186/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 149 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Contrastive Learning of Structured World Models](https://mlsanity.com/paper/1911.12247/) (2019)
- [GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2001.06216/) (2020)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [On the equivalence between graph isomorphism testing and function approximation with GNNs](https://mlsanity.com/paper/1905.12560/) (2019)
- [A Survey on The Expressive Power of Graph Neural Networks](https://mlsanity.com/paper/2003.04078/) (2020)
- [GMNN: Graph Markov Neural Networks](https://mlsanity.com/paper/1905.06214/) (2019)
- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders](https://mlsanity.com/paper/2201.05136/) (2022)
- [On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence](https://mlsanity.com/paper/2207.04630/) (2022)
- [On Particle Methods for Parameter Estimation in State-Space Models](https://mlsanity.com/paper/1412.8695/) (2014)
- [Gradient Dynamics of Shallow Univariate ReLU Networks](https://mlsanity.com/paper/1906.07842/) (2019)
- [Inference in Probabilistic Graphical Models by Graph Neural Networks](https://mlsanity.com/paper/1803.07710/) (2018)
- [Linear dynamical neural population models through nonlinear embeddings](https://mlsanity.com/paper/1605.08454/) (2016)
- [Active Learning of Dynamics for Data-Driven Control Using Koopman Operators](https://mlsanity.com/paper/1906.05194/) (2019)
- [Black box variational inference for state space models](https://mlsanity.com/paper/1511.07367/) (2015)
- [Learning interpretable continuous-time models of latent stochastic dynamical systems](https://mlsanity.com/paper/1902.04420/) (2019)
- [Capsule Network Performance on Complex Data](https://mlsanity.com/paper/1712.03480/) (2017)
- [Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics](https://mlsanity.com/paper/1908.04696/) (2019)
- [Factor Graph Neural Networks](https://mlsanity.com/paper/2308.00887/) (2023)
- [Shallow Univariate ReLu Networks as Splines: Initialization, Loss Surface, Hessian, & Gradient Flow Dynamics](https://mlsanity.com/paper/2008.01772/) (2020)
- [The implications of perception as probabilistic inference for correlated neural variability during behavior](https://mlsanity.com/paper/1409.0257/) (2014)
- [Flexible and accurate inference and learning for deep generative models](https://mlsanity.com/paper/1805.11051/) (2018)
- [Nonlinear System Identification of Soft Robot Dynamics Using Koopman Operator Theory](https://mlsanity.com/paper/1810.06637/) (2018)
- [The population doctrine in cognitive neuroscience](https://mlsanity.com/paper/2104.00145/) (2021)

## Built on by (most cited first, 1 of 1)

- [Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2410.17373/) (2024)

## Abstract (arXiv)

Patterns of microcircuitry suggest that the brain has an array of repeated canonical computational units. Yet neural representations are distributed, so the relevant computations may only be related indirectly to single-neuron transformations. It thus remains an open challenge how to define canonical distributed computations. We integrate normative and algorithmic theories of neural computation into a mathematical framework for inferring canonical distributed computations from large-scale neural activity patterns. At the normative level, we hypothesize that the brain creates a structured internal model of its environment, positing latent causes that explain its sensory inputs, and uses those sensory inputs to infer the latent causes. At the algorithmic level, we propose that this inference process is a nonlinear message-passing algorithm on a graph-structured model of the world. Given a time series of neural activity during a perceptual inference task, our framework finds (i) the neural representation of relevant latent variables, (ii) interactions between these variables that define the brain's internal model of the world, and (iii) message-functions specifying the inference algorithm. These targeted computational properties are then statistically distinguishable due to the symmetries inherent in any canonical computation, up to a global transformation. As a demonstration, we simulate recordings for a model brain that implicitly implements an approximate inference algorithm on a probabilistic graphical model. Given its external inputs and noisy neural activity, we recover the latent variables, their neural representation and dynamics, and canonical message-functions. We highlight features of experimental design needed to successfully extract canonical computations from neural data. Overall, this framework provides a new tool for discovering interpretable structure in neural recordings.