mlsanityData to 14 Oct 2026

Goal Discovery with Causal Capacity for Efficient Reinforcement Learning

arXiv 2508.09624 ↗First posted 13 Aug 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Causal inference is crucial for humans to explore the world, which can be modeled to enable an agent to efficiently explore the environment in reinforcement learning. Existing research indicates that establishing the causality between action and state transition will enhance an agent to reason how a policy affects its future trajectory, thereby promoting directed exploration. However, it is challenging to measure the causality due to its intractability in the vast state-action space of complex scenarios. In this paper, we propose a novel Goal Discovery with Causal Capacity (GDCC) framework for efficient environment exploration. Specifically, we first derive a measurement of causality in state space, i.e., causal capacity, which represents the highest influence of an agent's behavior on future trajectories. After that, we present a Monte Carlo based method to identify critical points in discrete state space and further optimize this method for continuous high-dimensional environments. Those critical points are used to uncover where the agent makes important decisions in the environment, which are then regarded as our subgoals to guide the agent to make exploration more purposefully and efficiently. Empirical results from multi-objective tasks demonstrate that states with high causal capacity align with our expected subgoals, and our GDCC achieves significant success rate improvements compared to baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 43 references that are in the map. First posted on arXiv 13 Aug 2025.

Builds on

25 of 43 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  7. 2018Exploration by Random Network Distillationcited by 993
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  10. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  12. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
13 more
  1. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  2. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
  3. 2020Invariant Causal Prediction for Block MDPscited by 105
  4. 2018Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Searchcited by 82
  5. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
  6. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  7. 2021Hierarchical Skills for Efficient Explorationcited by 38
  8. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34
  9. 2022Causal Imitation Learning under Temporally Correlated Noisecited by 28
  10. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 24
  11. 2023Causal Reinforcement Learning: A Surveycited by 23
  12. 2023Stochastic Graph Neural Network-based Value Decomposition for MARL in Internet of Vehiclescited by 7
  13. 2021Low rattling: A predictive principle for self-organization in active collectivescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Boosting Efficiency in Task-Agnostic Exploration through Causal Knowledge
  2. 2024Variable-Agnostic Causal Exploration for Reinforcement Learning
  3. 2024Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy
  4. 2025Causal Information Prioritization for Efficient Reinforcement Learning
  5. 2025Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challenges
  6. 2025Reducing Action Space for Deep Reinforcement Learning via Causal Effect Estimation
  7. 2026CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning
  8. 2025Hierarchical Reinforcement Learning with Targeted Causal Interventions