mlsanityData to 14 Oct 2026

Causality-Based Reinforcement Learning Method for Multi-Stage Robotic Tasks

arXiv 2503.03145 ↗First posted 5 Mar 2025cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Deep reinforcement learning has made significant strides in various robotic tasks. However, employing deep reinforcement learning methods to tackle multi-stage tasks still a challenge. Reinforcement learning algorithms often encounter issues such as redundant exploration, getting stuck in dead ends, and progress reversal in multi-stage tasks. To address this, we propose a method that integrates causal relationships with reinforcement learning for multi-stage tasks. Our approach enables robots to automatically discover the causal relationships between their actions and the rewards of the tasks and constructs the action space using only causal actions, thereby reducing redundant exploration and progress reversal. By integrating correct causal relationships using the causal policy gradient method into the learning process, our approach can enhance the performance of reinforcement learning algorithms in multi-stage robotic tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 38 references that are in the map. First posted on arXiv 5 Mar 2025.

Builds on

21 of 38 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  5. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  6. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  7. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 194
  8. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  9. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
  10. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 55
  11. 2023A Survey on Causal Reinforcement Learningcited by 45
  12. 2021Causal Reinforcement Learning using Observational and Interventional Datacited by 39
9 more
  1. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 24
  2. 2021Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policiescited by 22
  3. 2023Causal Policy Gradient for Whole-Body Mobile Manipulationcited by 21
  4. 2019Vision-based Navigation Using Deep Reinforcement Learningcited by 13
  5. 2023Multi-Stage Reinforcement Learning for Non-Prehensile Manipulationcited by 7
  6. 2023Hierarchical Visual Policy Learning for Long-Horizon Robot Manipulation in Densely Cluttered Scenescited by 5
  7. 2022Developing cooperative policies for multi-stage reinforcement learning taskscited by 4
  8. 2021Learning Robotic Manipulation Tasks via Task Progress based Gaussian Reward and Loss Adjusted Explorationcited by 2
  9. 2022Learning Multi-step Robotic Manipulation Policies from Visual Observation of Scene and Q-value Predictions of Previous Actioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Learning Transition Models with Time-delayed Causal Relations
  2. 2026CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning
  3. 2025Causal-Paced Deep Reinforcement Learning
  4. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
  5. 2025Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies
  6. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review
  7. 2026Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
  8. 2024Causal Reinforcement Learning for Optimisation of Robot Dynamics in Unknown Environments