mlsanityData to 7 Oct 2026

Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning

arXiv 2405.03113 ↗First posted 6 May 2024cs.RO cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning is a promising tool for learning complex policies even in fast-moving and object-interactive domains where human teleoperation or hard-coded policies might fail. To effectively reflect this challenging category of tasks, we introduce a dynamic, interactive RL testbed based on robot air hockey. By augmenting air hockey with a large family of tasks ranging from easy tasks like reaching, to challenging ones like pushing a block by hitting it with a puck, as well as goal-based and human-interactive tasks, our testbed allows a varied assessment of RL capabilities. The robot air hockey testbed also supports sim-to-real transfer with three domains: two simulators of increasing fidelity and a real robot system. Using a dataset of demonstration data gathered through two teleoperation systems: a virtualized control environment, and human shadowing, we assess the testbed with behavior cloning, offline RL, and RL from scratch.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

30 of 59 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  9. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  10. 2017Hindsight Experience Replaycited by 1,096
  11. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  12. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
18 more
  1. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  2. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  3. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  4. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  5. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  6. 2023Contrastive Preference Learning: Learning from Human Feedback without RLcited by 77
  7. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  8. 2021DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularizationcited by 66
  9. 2023Dual RL: Unification and New Methods for Reinforcement and Imitation Learningcited by 48
  10. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  11. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  12. 2023What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discoverycited by 22
  13. 2023SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learningcited by 21
  14. 2022A Ranking Game for Imitation Learningcited by 18
  15. 2019Hypothesis-Driven Skill Discovery for Hierarchical Deep Reinforcement Learningcited by 8
  16. 2021Efficient and Reactive Planning for High Speed Robot Air Hockeycited by 8
  17. 2017Learning Control for Air Hockey Striking using Deep Reinforcement Learningcited by 3
  18. 2024AiRLIHockey: Highly Reactive Contact Control and Stochastic Optimal Shootingcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasetscited by 10
  2. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  3. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 4
  4. 2024A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Roboticscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations
  2. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
  3. 2025X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real
  4. 2024Learning Diverse Robot Striking Motions with Diffusion Models and Kinematically Constrained Gradient Guidance
  5. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration
  6. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  7. 2022How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
  8. 2026One Demonstration Is Enough for Real-World Robotic Reinforcement Learning ↗