Amortized Active Causal Induction with Deep Reinforcement Learning

arXiv 2405.16718 ↗First posted 26 May 2024cs.LG cs.AIcited by 10

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

We present Causal Amortized Active Structure Learning (CAASL), an active intervention design policy that can select interventions that are adaptive, real-time and that does not require access to the likelihood. This policy, an amortized network based on the transformer, is trained with reinforcement learning on a simulator of the design environment, and a reward function that measures how close the true causal graph is to a causal graph posterior inferred from the gathered data. On synthetic data and a single-cell gene expression simulator, we demonstrate empirically that the data acquired through our policy results in a better estimate of the underlying causal graph than alternative strategies. Our design policy successfully achieves amortized intervention design on the distribution of the training environment while also generalizing well to distribution shifts in test-time design environments. Further, our policy also demonstrates excellent zero-shot generalization to design environments with dimensionality higher than that during training, and to intervention types that it has not been trained on.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 37 of its 61 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

37 of 61 references.

Closest ancestors · 1

Same direction, newest first

2024CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning30 Jan

Other references · 36

2017Attention Is All You Needcited by 60,361
2020Language Models are Few-Shot Learnerscited by 32,419
2020Scaling Laws for Neural Language Modelscited by 6,586
2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
2021Scaling Vision Transformerscited by 999
2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
2020Differentiable Causal Discovery from Interventional Datacited by 218
2021Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Gamecited by 161
2019Causal Discovery with Reinforcement Learningcited by 145
2023Modern Bayesian Experimental Designcited by 125
2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
2022Score matching enables causal discovery of nonlinear additive noise modelscited by 111

24 more

2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 106
2022Amortized Inference for Causal Structure Learningcited by 90
2017Causal Structure Learningcited by 84
2013Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizationscited by 70
2022Learning to Induce Causal Structurecited by 68
2021Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoodscited by 61
2019A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experimentscited by 58
2019ABCD-Strategy: Budgeted Experimental Design for Targeted Causal Structure Discoverycited by 52
2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
2022Interventions, Where and How? Experimental Design for Causal Models at Scalecited by 49
2022Optimizing Sequential Experimental Design with Deep Reinforcement Learningcited by 46
2015backShift: Learning causal cyclic graphs from unknown shift interventionscited by 37
2013Data Analysis with Bayesian Networks: A Bootstrap Approachcited by 32
2016Sequential Bayesian optimal experimental design via approximate dynamic programmingcited by 28
2023Assumption violations in causal discovery and the robustness of score matchingcited by 28
2023Causal machine learning for single-cell genomicscited by 23
2023Passive learning of active causal strategies in agents and language modelscited by 21
2021Near-Optimal Multi-Perturbation Experimental Design for Causal Structure Learningcited by 17
2023Differentiable Multi-Target Causal Bayesian Experimental Designcited by 15
2024The Causal Chambers: Real Physical Systems as a Testbed for AI Methodologycited by 12
2022A Meta-Reinforcement Learning Algorithm for Causal Discoverycited by 11
2024Gene Regulatory Network Inference in the Presence of Dropouts: a Causal Viewcited by 11
2023Adaptivity Complexity for Causal Graph Discoverycited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

2025Goal-Oriented Sequential Bayesian Experimental Design for Causal Learningcited by 2
2026CDFM: Towards a General-Purpose Causal Discovery Foundation Modelcited by 2
2026Can Neural Networks Learn by Experimenting on Themselves? Self-Interventional Learning from Functional Consequences to Predictive Self-Knowledgecited by 2
2025ACTIVA: Amortized Causal Effect Estimation via Transformer-based Variational Autoencodercited by 1
2025ActiveCQ: Active Estimation of Causal Quantitiescited by 1
2026Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learningcited by 1
2026DCD-PFN: A Decoupling-Aware Foundation Model for Causal Discoverycited by 1
2026Causal Preference Elicitationcited by 0
2026CausalNav: Reliability-Certified Causal World Models for Control under Physical-Parameter Shiftcited by 0
2026Certifying Interventional Agreement Among Observationally Equivalent Causal Modelscited by 0

Similar, not cited

Close in wording, no citation either way