# A Refined Analysis of UCBVI

arXiv: https://arxiv.org/abs/2502.17370 · HTML: https://mlsanity.com/paper/2502.17370/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 9 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 3 of 3)

- [Minimax Regret Bounds for Reinforcement Learning](https://mlsanity.com/paper/1703.05449/) (2017)
- [Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisited](https://mlsanity.com/paper/2010.03531/) (2020)
- [Settling the Sample Complexity of Online Reinforcement Learning](https://mlsanity.com/paper/2307.13586/) (2023)

## Built on by (most cited first, 1 of 1)

- [Learning in Markov Decision Processes with Exogenous Dynamics](https://mlsanity.com/paper/2603.02862/) (2026)

## Abstract (arXiv)

In this work, we provide a refined analysis of the UCBVI algorithm (Azar et al., 2017), improving both the bonus terms and the regret analysis. Additionally, we compare our version of UCBVI with both its original version and the state-of-the-art MVP algorithm. Our empirical validation demonstrates that improving the multiplicative constants in the bounds has significant positive effects on the empirical performance of the algorithms.