# The LoCA Regret: A Consistent Metric to Evaluate Model-Based Behavior in Reinforcement Learning

arXiv: https://arxiv.org/abs/2007.03158 · HTML: https://mlsanity.com/paper/2007.03158/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [The Effectiveness of World Models for Continual Reinforcement Learning](https://mlsanity.com/paper/2211.15944/) (2022)
- [Curious Replay for Model-based Adaptation](https://mlsanity.com/paper/2306.15934/) (2023)
- [Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods](https://mlsanity.com/paper/2204.11464/) (2022)
- [Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi](https://mlsanity.com/paper/2308.10284/) (2023)
- [Visualizing MuZero Models](https://mlsanity.com/paper/2102.12924/) (2021)
- [A Rubric for Human-like Agents and NeuroAI](https://mlsanity.com/paper/2212.04401/) (2022)
- [Replay Buffer with Local Forgetting for Adapting to Local Environment Changes in Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/2303.08690/) (2023)
- [A Look at Value-Based Decision-Time vs. Background Planning Methods Across Different Settings](https://mlsanity.com/paper/2206.08442/) (2022)
- [How does Your RL Agent Explore? An Optimal Transport Analysis of Occupancy Measure Trajectories](https://mlsanity.com/paper/2402.09113/) (2024)
- [Planning with Expectation Models for Control](https://mlsanity.com/paper/2104.08543/) (2021)
- [Partial Models for Building Adaptive Model-Based Reinforcement Learning Agents](https://mlsanity.com/paper/2405.16899/) (2024)

## Abstract (arXiv)

Deep model-based Reinforcement Learning (RL) has the potential to substantially improve the sample-efficiency of deep RL. While various challenges have long held it back, a number of papers have recently come out reporting success with deep model-based methods. This is a great development, but the lack of a consistent metric to evaluate such methods makes it difficult to compare various approaches. For example, the common single-task sample-efficiency metric conflates improvements due to model-based learning with various other aspects, such as representation learning, making it difficult to assess true progress on model-based RL. To address this, we introduce an experimental setup to evaluate model-based behavior of RL methods, inspired by work from neuroscience on detecting model-based behavior in humans and animals. Our metric based on this setup, the Local Change Adaptation (LoCA) regret, measures how quickly an RL method adapts to a local change in the environment. Our metric can identify model-based behavior, even if the method uses a poor representation and provides insight in how close a method's behavior is from optimal model-based behavior. We use our setup to evaluate the model-based behavior of MuZero on a variation of the classic Mountain Car task.