mlsanityData to 7 Oct 2026

Thinking While Moving: Deep Reinforcement Learning with Concurrent Control

arXiv 2004.06089 ↗First posted 13 Apr 2020cs.LG cs.AI cs.ROcited by 29

Not on the map. Why?

Abstract (arXiv)

We study reinforcement learning in settings where sampling an action from the policy must be done concurrently with the time evolution of the controlled system, such as when a robot must decide on the next action while still performing the previous action. Much like a person or an animal, the robot must think and move at the same time, deciding on its next action before the previous one has completed. In order to develop an algorithmic framework for such concurrent control problems, we start with a continuous-time formulation of the Bellman equations, and then discretize them in a way that is aware of system delays. We instantiate this new class of approximate dynamic programming methods via a simple architectural extension to existing value-based deep reinforcement learning algorithms. We evaluate our methods on simulated benchmark tasks and a large-scale robotic grasping task where the robot must "think while moving".

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 2; 2024: 1; 2025: 3; 2026: 7.

29 papers.

Sort:

By area:

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  3. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  4. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  5. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  6. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  7. 2021Unsupervised Domain Adaptation of Object Detectors: A Surveycited by 93
  8. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  9. 2021Acting in Delayed Environments with Non-Stationary Markov Policiescited by 28
  10. 2022Token Turing Machinescited by 25
  11. 2021Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delayscited by 20
  12. 2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 18
17 more
  1. 2023DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inferencecited by 13
  2. 2022Dynamic Decision Frequency with Continuous Optionscited by 11
  3. 2024DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delayscited by 6
  4. 2022PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scalecited by 5
  5. 2022Multiscale Sensor Fusion and Continuous Control with Neural CDEscited by 4
  6. 2022Offline Reinforcement Learning at Multiple Frequenciescited by 4
  7. 2026DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executorscited by 3
  8. 2026Finding the Time to Think: Learning Planning Budgets in Real-Time RLcited by 2
  9. 2022Off-Beat Multi-Agent Reinforcement Learningcited by 1
  10. 2025Learning to Recover: Dynamic Reward Shaping with Wheel-Leg Coordination for Fallen Robotscited by 1
  11. 2026Latency-Tolerant Cloud-Edge Collaborative Vision-Language-Action Models via Emergent Representational Specializationcited by 1
  12. 2026Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latencycited by 1
  13. 2022Non-Parametric Stochastic Policy Gradient with Strategic Retreat for Non-Stationary Environmentcited by 0
  14. 2025RT-HCP: Dealing with Inference Delays and Sample Efficiency to Learn Directly on Robotic Platformscited by 0
  15. 2026Real-Time Execution with Autoregressive Policiescited by 0
  16. 2026Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Drivingcited by 0
  17. 2026World Models Under Asynchronous Sensor Observationscited by 0