# A Direct Approximation of AIXI Using Logical State Abstractions

arXiv: https://arxiv.org/abs/2210.06917 · HTML: https://mlsanity.com/paper/2210.06917/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 50 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 7 of 7)

- [Epidemic processes in complex networks](https://mlsanity.com/paper/1408.2701/) (2014)
- [Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks](https://mlsanity.com/paper/2010.05313/) (2020)
- [Reinforcement Learning for Optimization of COVID-19 Mitigation policies](https://mlsanity.com/paper/2010.10560/) (2020)
- [Analysis and Control of Epidemics: A survey of spreading processes on complex networks](https://mlsanity.com/paper/1505.00768/) (2015)
- [EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Models](https://mlsanity.com/paper/2010.04452/) (2020)
- [COVID-19 Pandemic Cyclic Lockdown Optimization Using Reinforcement Learning](https://mlsanity.com/paper/2009.04647/) (2020)
- [COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability](https://mlsanity.com/paper/2005.06526/) (2020)

## Built on by (most cited first, 2 of 2)

- [Generative AI vs. AGI: The Cognitive Strengths and Weaknesses of Modern LLMs](https://mlsanity.com/paper/2309.10371/) (2023)
- [Dynamic Knowledge Injection for AIXI Agents](https://mlsanity.com/paper/2312.16184/) (2023)

## Abstract (arXiv)

We propose a practical integration of logical state abstraction with AIXI, a Bayesian optimality notion for reinforcement learning agents, to significantly expand the model class that AIXI agents can be approximated over to complex history-dependent and structured environments. The state representation and reasoning framework is based on higher-order logic, which can be used to define and enumerate complex features on non-Markovian and structured environments. We address the problem of selecting the right subset of features to form state abstractions by adapting the Phi-MDP optimisation criterion from state abstraction theory. Exact Bayesian model learning is then achieved using a suitable generalisation of Context Tree Weighting over abstract state sequences. The resultant architecture can be integrated with different planning algorithms. Experimental results on controlling epidemics on large-scale contact networks validates the agent's performance.