# A Probabilistic Generative Model of Free Categories

arXiv: https://arxiv.org/abs/2205.04545 · HTML: https://mlsanity.com/paper/2205.04545/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 75 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 30 of 34)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [Neuro-Symbolic Program Synthesis](https://mlsanity.com/paper/1611.01855/) (2016)
- [Learning Compositional Rules via Neural Program Synthesis](https://mlsanity.com/paper/2003.05562/) (2020)
- [An Introduction to Probabilistic Programming](https://mlsanity.com/paper/1809.10756/) (2018)
- [Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning](https://mlsanity.com/paper/2010.00763/) (2020)
- [A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics](https://mlsanity.com/paper/1908.07021/) (2019)
- [The Omniglot challenge: a 3-year progress report](https://mlsanity.com/paper/1902.03477/) (2019)
- [One-Shot Generalization in Deep Generative Models](https://mlsanity.com/paper/1603.05106/) (2016)
- [Backprop as Functor: A compositional perspective on supervised learning](https://mlsanity.com/paper/1711.10455/) (2017)
- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [HOUDINI: Lifelong Learning as Program Synthesis](https://mlsanity.com/paper/1804.00218/) (2018)
- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Disintegration and Bayesian Inversion via String Diagrams](https://mlsanity.com/paper/1709.00322/) (2017)
- [Faithful Inversion of Generative Models for Effective Amortized Inference](https://mlsanity.com/paper/1712.00287/) (2017)
- [Sub-Goal Trees -- a Framework for Goal-Based Reinforcement Learning](https://mlsanity.com/paper/2002.12361/) (2020)
- [Learning Hierarchical Features from Generative Models](https://mlsanity.com/paper/1702.08396/) (2017)
- [The Variational Homoencoder: Learning to learn high capacity generative models from few examples](https://mlsanity.com/paper/1807.08919/) (2018)
- [People infer recursive visual concepts from just a few examples](https://mlsanity.com/paper/1904.08034/) (2019)
- [A Convenient Category for Higher-Order Probability Theory](https://mlsanity.com/paper/1701.02547/) (2017)
- [Learning Task-General Representations with Generative Neuro-Symbolic Modeling](https://mlsanity.com/paper/2006.14448/) (2020)
- [Lenses and Learners](https://mlsanity.com/paper/1903.03671/) (2019)
- [The operad of wiring diagrams: formalizing a graphical language for databases, recursion, and plug-and-play circuits](https://mlsanity.com/paper/1305.0297/) (2013)
- [Denotational validation of higher-order Bayesian inference](https://mlsanity.com/paper/1711.03219/) (2017)
- [Learning Functors using Gradient Descent](https://mlsanity.com/paper/2009.06837/) (2020)
- [Categorical Data Structures for Technical Computing](https://mlsanity.com/paper/2106.04703/) (2021)
- [A metric on directed graphs and Markov chains based on hitting probabilities](https://mlsanity.com/paper/2006.14482/) (2020)
- [The operad of temporal wiring diagrams: formalizing a graphical language for discrete-time processes](https://mlsanity.com/paper/1307.6894/) (2013)

## Abstract (arXiv)

Applied category theory has recently developed libraries for computing with morphisms in interesting categories, while machine learning has developed ways of learning programs in interesting languages. Taking the analogy between categories and languages seriously, this paper defines a probabilistic generative model of morphisms in free monoidal categories over domain-specific generating objects and morphisms. The paper shows how acyclic directed wiring diagrams can model specifications for morphisms, which the model can use to generate morphisms. Amortized variational inference in the generative model then enables learning of parameters (by maximum likelihood) and inference of latent variables (by Bayesian inversion). A concrete experiment shows that the free category prior achieves competitive reconstruction performance on the Omniglot dataset.