# Dynamic Knowledge Injection for AIXI Agents

arXiv: https://arxiv.org/abs/2312.16184 · HTML: https://mlsanity.com/paper/2312.16184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 39 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 7 of 7)

- [Objective Mismatch in Model-based Reinforcement Learning](https://mlsanity.com/paper/2002.04523/) (2020)
- [Epidemic processes in complex networks](https://mlsanity.com/paper/1408.2701/) (2014)
- [Analysis and Control of Epidemics: A survey of spreading processes on complex networks](https://mlsanity.com/paper/1505.00768/) (2015)
- [EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Models](https://mlsanity.com/paper/2010.04452/) (2020)
- [Efficient tracking of a growing number of experts](https://mlsanity.com/paper/1708.09811/) (2017)
- [COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability](https://mlsanity.com/paper/2005.06526/) (2020)
- [A Direct Approximation of AIXI Using Logical State Abstractions](https://mlsanity.com/paper/2210.06917/) (2022)

## Abstract (arXiv)

Prior approximations of AIXI, a Bayesian optimality notion for general reinforcement learning, can only approximate AIXI's Bayesian environment model using an a-priori defined set of models. This is a fundamental source of epistemic uncertainty for the agent in settings where the existence of systematic bias in the predefined model class cannot be resolved by simply collecting more data from the environment. We address this issue in the context of Human-AI teaming by considering a setup where additional knowledge for the agent in the form of new candidate models arrives from a human operator in an online fashion. We introduce a new agent called DynamicHedgeAIXI that maintains an exact Bayesian mixture over dynamically changing sets of models via a time-adaptive prior constructed from a variant of the Hedge algorithm. The DynamicHedgeAIXI agent is the richest direct approximation of AIXI known to date and comes with good performance guarantees. Experimental results on epidemic control on contact networks validates the agent's practical utility.