mlsanityData to 14 Oct 2026

RT-HCP: Dealing with Inference Delays and Sample Efficiency to Learn Directly on Robotic Platforms

arXiv 2509.06714 ↗First posted 8 Sep 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Learning a controller directly on the robot requires extreme sample efficiency. Model-based reinforcement learning (RL) methods are the most sample efficient, but they often suffer from a too long inference time to meet the robot control frequency requirements. In this paper, we address the sample efficiency and inference time challenges with two contributions. First, we define a general framework to deal with inference delays where the slow inference robot controller provides a sequence of actions to feed the control-hungry robotic platform without execution gaps. Then, we compare several RL algorithms in the light of this framework and propose RT-HCP, an algorithm that offers an excellent trade-off between performance, sample efficiency and inference time. We validate the superiority of RT-HCP with experiments where we learn a controller directly on a simple but high frequency FURUTA pendulum platform. Code: github.com/elasriz/RTHCP

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

15 of 21 references.

References · 15

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  3. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  4. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  5. 2019Challenges of Real-World Reinforcement Learningcited by 291
  6. 2021Mastering Atari Games with Limited Datacited by 237
  7. 2019Exploring Model-based Planning with Policy Networkscited by 90
  8. 2020Reinforcement Learning with Random Delayscited by 59
  9. 2019Real-Time Reinforcement Learningcited by 40
  10. 2020Delay-Aware Model-Based Reinforcement Learning for Continuous Controlcited by 34
  11. 2020Blending MPC & Value Function Approximation for Efficient Reinforcement Learningcited by 32
  12. 2020Thinking While Moving: Deep Reinforcement Learning with Concurrent Controlcited by 31
3 more
  1. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 22
  2. 2024Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inferencecited by 7
  3. 2024Physics-Informed Model and Hybrid Planning for Efficient Dyna-Style Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
  2. 2026Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency
  3. 2024Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control
  4. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review
  5. 2025Model-Based Reinforcement Learning under Random Observation Delays
  6. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  7. 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Exploration
  8. 2024RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Models