# Efficient search of active inference policy spaces using k-means

arXiv: https://arxiv.org/abs/2209.02550 · HTML: https://mlsanity.com/paper/2209.02550/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 6 Sep 2022.

## Builds on (most cited first, 12 of 12)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Optimizing the Latent Space of Generative Networks](https://mlsanity.com/paper/1707.05776/) (2017)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Deep Successor Reinforcement Learning](https://mlsanity.com/paper/1606.02396/) (2016)
- [Deep active inference agents using Monte-Carlo methods](https://mlsanity.com/paper/2006.04176/) (2020)
- [On the Relationship Between Active Inference and Control as Inference](https://mlsanity.com/paper/2006.12964/) (2020)
- [pymdp: A Python library for active inference in discrete state spaces](https://mlsanity.com/paper/2201.03904/) (2022)
- [Branching Time Active Inference: the theory and its generality](https://mlsanity.com/paper/2111.11107/) (2021)
- [Branching Time Active Inference: empirical study and complexity class analysis](https://mlsanity.com/paper/2111.11276/) (2021)
- [Hierarchical reinforcement learning for efficient exploration and transfer](https://mlsanity.com/paper/2011.06335/) (2020)
- [Successor Representation Active Inference](https://mlsanity.com/paper/2207.09897/) (2022)

## Abstract (arXiv)

We develop an approach to policy selection in active inference that allows us to efficiently search large policy spaces by mapping each policy to its embedding in a vector space. We sample the expected free energy of representative points in the space, then perform a more thorough policy search around the most promising point in this initial sample. We consider various approaches to creating the policy embedding space, and propose using k-means clustering to select representative points. We apply our technique to a goal-oriented graph-traversal problem, for which naive policy selection is intractable for even moderately large graphs.