# Learning to Play Air Hockey with Model-Based Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2406.00518 · HTML: https://mlsanity.com/paper/2406.00518/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 1 of its 4 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 1 of 1)

- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)

## Built on by (most cited first, 2 of 2)

- [NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Games](https://mlsanity.com/paper/2609.01549/) (2026)
- [Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey](https://mlsanity.com/paper/2609.39151/) (2026)

## Abstract (arXiv)

In the context of addressing the Robot Air Hockey Challenge 2023, we investigate the applicability of model-based deep reinforcement learning to acquire a policy capable of autonomously playing air hockey. Our agents learn solely from sparse rewards while incorporating self-play to iteratively refine their behaviour over time. The robotic manipulator is interfaced using continuous high-level actions for position-based control in the Cartesian plane while having partial observability of the environment with stochastic transitions. We demonstrate that agents are prone to overfitting when trained solely against a single playstyle, highlighting the importance of self-play for generalization to novel strategies of unseen opponents. Furthermore, the impact of the imagination horizon is explored in the competitive setting of the highly dynamic game of air hockey, with longer horizons resulting in more stable learning and better overall performance.