mlsanityData to 14 Oct 2026

Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challenges

arXiv 2505.08453 ↗First posted 13 May 2025cs.RO cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Causal understanding is important in many disciplines of science and engineering, where we seek to understand how different factors in the system causally affect an experiment or situation and pave a pathway towards creating effective or optimising existing models. Examples of use cases are autonomous exploration and modelling of unknown environments or assessing key variables in optimising large complex systems. In this paper, we analyse a Reinforcement Learning approach called Causal Curiosity, which aims to estimate as accurately and efficiently as possible, without directly measuring them, the value of factors that causally determine the dynamics of a system. Whilst the idea presents a pathway forward, measurement accuracy is the foundation of methodology effectiveness. Focusing on the current causal curiosity's robotic manipulator, we present for the first time a measurement accuracy analysis of the future potentials and current limitations of this technique and an analysis of its sensitivity and confounding factor disentanglement capability - crucial for causal analysis. As a result of our work, we promote proposals for an improved and efficient design of Causal Curiosity methods to be applied to real-world complex scenarios.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 49 of its 89 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

49 of 89 references.

References · 49

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  3. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  4. 2017Deep Reinforcement Learning that Matterscited by 754
  5. 2018Recognition in Terra Incognitacited by 658
  6. 2016Deep Exploration via Bootstrapped DQNcited by 527
  7. 2018Generating Natural Language Adversarial Examplescited by 424
  8. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  9. 2019Causal Confusion in Imitation Learningcited by 288
  10. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  11. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 271
  12. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
37 more
  1. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 211
  2. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  3. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
  4. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  5. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  6. 2022Causal Machine Learning: A Survey and Open Problemscited by 110
  7. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  8. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  9. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 83
  10. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
  11. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
  12. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 62
  13. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 59
  14. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58
  15. 2019Causal Reasoning from Meta-reinforcement Learningcited by 53
  16. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  17. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  18. 2023A Survey on Causal Reinforcement Learningcited by 45
  19. 2018GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithmscited by 40
  20. 2021Causal Reinforcement Learning using Observational and Interventional Datacited by 39
  21. 2018Deconfounding Reinforcement Learning in Observational Settingscited by 36
  22. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34
  23. 2019Causal Induction from Visual Observations for Goal Directed Taskscited by 31
  24. 2020TriFinger: An Open-Source Robot for Learning Dexteritycited by 31
  25. 2020Should Graph Convolution Trust Neighbors? A Simple Causal Inference Methodcited by 25
  26. 2020Causally Correct Partial Models for Reinforcement Learningcited by 24
  27. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 24
  28. 2018Generalization in anti-causal learningcited by 23
  29. 2017Shape-independent Hardness Estimation Using Deep Learning and a GelSight Tactile Sensorcited by 19
  30. 2018Learning to Grasp Without Seeingcited by 19
  31. 2018Curiosity Driven Exploration of Learned Disentangled Goal Spacescited by 16
  32. 2021Causal Multi-Agent Reinforcement Learning: Review and Open Problemscited by 16
  33. 2020Resolving Spurious Correlations in Causal Models of Environments via Interventionscited by 14
  34. 2018Computational Theories of Curiosity-Driven Learningcited by 11
  35. 2022Hyperparameter Tuning for Deep Reinforcement Learning Applicationscited by 7
  36. 2022Learning by Doing: Controlling a Dynamical System using Causality, Control, and Reinforcement Learningcited by 7
  37. 2019Estimation and Exploitation of Objects' Inertial Parameters in Robotic Grasping and Manipulation: A Surveycited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Variable-Agnostic Causal Exploration for Reinforcement Learning
  2. 2024Boosting Efficiency in Task-Agnostic Exploration through Causal Knowledge
  3. 2024Causal Reinforcement Learning for Optimisation of Robot Dynamics in Unknown Environments
  4. 2025Goal Discovery with Causal Capacity for Efficient Reinforcement Learning
  5. 2026An Introduction to Causal Reinforcement Learning
  6. 2025Causal Information Prioritization for Efficient Reinforcement Learning
  7. 2025Curiosity-Diffuser: Curiosity Guide Diffusion Models for Reliability
  8. 2024Rethinking State Disentanglement in Causal Reinforcement Learning