# Reinforcement Learning: An Overview

arXiv: https://arxiv.org/abs/2412.05265 · HTML: https://mlsanity.com/paper/2412.05265/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 0 of its 2 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)
- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Built on by (most cited first, 7 of 7)

- [Code World Models for General Game Playing](https://mlsanity.com/paper/2510.04542/) (2025)
- [More Bang for the Buck: Process Reward Modeling with Entropy-Driven Uncertainty](https://mlsanity.com/paper/2503.22233/) (2025)
- [Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling](https://mlsanity.com/paper/2510.05825/) (2025)
- [Action-BED: Task-Driven Bayesian Experimental Design with Singly Intractable Objectives](https://mlsanity.com/paper/2606.23662/) (2026)
- [Fully Offline Reinforcement Learning](https://mlsanity.com/paper/2505.22442/) (2025)
- [MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning](https://mlsanity.com/paper/2607.24064/) (2026)
- [World models of environment, agent and joint agent-environment systems](https://mlsanity.com/paper/2608.20401/) (2026)

## Abstract (arXiv)

This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based methods, policy-based methods, model-based methods, multi-agent RL, LLMs and RL, and various other topics (e.g., offline RL, hierarchical RL, intrinsic reward). It also includes some code snippets for training LLMs with RL.