# An Information-Theoretic Approach for Automatically Determining the Number of States when Aggregating Markov Chains

arXiv: https://arxiv.org/abs/2107.01799 · HTML: https://mlsanity.com/paper/2107.01799/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 29 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 4 of 4)

- [Optimal Kullback-Leibler Aggregation via Information Bottleneck](https://mlsanity.com/paper/1304.6603/) (2013)
- [Guided Policy Exploration for Markov Decision Processes using an Uncertainty-Based Value-of-Information Criterion](https://mlsanity.com/paper/1802.01518/) (2018)
- [Analysis of Agent Expertise in Ms. Pac-Man using Value-of-Information-based Policies](https://mlsanity.com/paper/1702.08628/) (2017)
- [An Analysis of the Value of Information when Exploring Stochastic, Discrete Multi-Armed Bandits](https://mlsanity.com/paper/1710.02869/) (2017)

## Abstract (arXiv)

A fundamental problem when aggregating Markov chains is the specification of the number of state groups. Too few state groups may fail to sufficiently capture the pertinent dynamics of the original, high-order Markov chain. Too many state groups may lead to a non-parsimonious, reduced-order Markov chain whose complexity rivals that of the original. In this paper, we show that an augmented value-of-information-based approach to aggregating Markov chains facilitates the determination of the number of state groups. The optimal state-group count coincides with the case where the complexity of the reduced-order chain is balanced against the mutual dependence between the original- and reduced-order chain dynamics.