# A Model-Free Universal AI

arXiv: https://arxiv.org/abs/2602.23242 · HTML: https://mlsanity.com/paper/2602.23242/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 67 references that are in the map.
- First posted on arXiv 26 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [A Definition of Continual Reinforcement Learning](https://mlsanity.com/paper/2307.11046/) (2023)
- [Self-Modification of Policy and Utility Function in Rational Agents](https://mlsanity.com/paper/1605.03142/) (2016)
- [Nonparametric General Reinforcement Learning](https://mlsanity.com/paper/1611.08944/) (2016)
- [A Strongly Asymptotically Optimal Agent in General Environments](https://mlsanity.com/paper/1903.01021/) (2019)
- [Embedded Universal Predictive Intelligence: a coherent framework for multi-agent learning](https://mlsanity.com/paper/2511.22226/) (2025)
- [Thompson Sampling is Asymptotically Optimal in General Environments](https://mlsanity.com/paper/1602.07905/) (2016)
- [Compress and Control](https://mlsanity.com/paper/1411.5326/) (2014)
- [A Formal Solution to the Grain of Truth Problem](https://mlsanity.com/paper/1609.05058/) (2016)
- [Curiosity Killed or Incapacitated the Cat and the Asymptotically Optimal Agent](https://mlsanity.com/paper/2006.03357/) (2020)
- [Categorizing Wireheading in Partially Embedded Agents](https://mlsanity.com/paper/1906.09136/) (2019)
- [Extreme State Aggregation Beyond MDPs](https://mlsanity.com/paper/1407.3341/) (2014)
- [Limit-Computable Grains of Truth for Arbitrary Computable Extensive-Form (Un)Known Games](https://mlsanity.com/paper/2508.16245/) (2025)

## Built on by (most cited first, 1 of 1)

- [From AGI to ASI](https://mlsanity.com/paper/2606.12683/) (2026)

## Abstract (arXiv)

In general reinforcement learning, all established optimal agents, including AIXI, are model-based, explicitly maintaining and using environment models. This paper introduces Universal AI with Q-Induction (AIQI), the first model-free agent proven to be asymptotically varepsilon-optimal in general RL. AIQI performs universal induction over distributional action-value functions, instead of policies or environments like previous works. Under a grain of truth condition, we prove that AIQI is strong asymptotically varepsilon-optimal and asymptotically varepsilon-Bayes-optimal. We also apply our novel proof techniques to show asymptotic varepsilon-optimality of Self-AIXI without any ad-hoc assumptions. Our results significantly expand the diversity of known universal agents.