mlsanityData to 7 Oct 2026

The Causal-Neural Connection: Expressiveness, Learnability, and Inference

arXiv 2107.00793 ↗First posted 2 Jul 2021cs.LG cs.AIcited by 97

Not on the map. Why?

Abstract (arXiv)

One of the central elements of any causal inference is an object called structural causal model (SCM), which represents a collection of mechanisms and exogenous sources of random variation of the system under investigation (Pearl, 2000). An important property of many kinds of neural networks is universal approximability: the ability to approximate any function to arbitrary precision. Given this property, one may be tempted to surmise that a collection of neural nets is capable of learning any SCM by training on data generated by that SCM. In this paper, we show this is not the case by disentangling the notions of expressivity and learnability. Specifically, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020), which describes the limits of what can be learned from data, still holds for neural models. For instance, an arbitrarily complex and expressive neural net is unable to predict the effects of interventions given observational data alone. Given this result, we introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences. Building on this new class of models, we focus on solving two canonical tasks found in the literature known as causal identification and estimation. Leveraging the neural toolbox, we develop an algorithm that is both sufficient and necessary to determine whether a causal effect can be learned from data (i.e., causal identifiability); it then estimates the effect whenever identifiability holds (causal estimation). Simulations corroborate the proposed approach.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 24 of its 75 references that are in the map. First posted on arXiv 2 Jul 2021.

Builds on

24 of 75 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2015Variational Inference with Normalizing Flowscited by 2,161
  7. 2016Variational Inference: A Review for Statisticianscited by 1,466
  8. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
  9. 2016Learning Representations for Counterfactual Inferencecited by 327
  10. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  11. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  12. 2019Adapting Neural Networks for the Estimation of Treatment Effectscited by 285
12 more
  1. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  2. 2020Differentiable Causal Discovery from Interventional Datacited by 218
  3. 2019A Meta-Transfer Objective for Learning to Disentangle Causal Mechanismscited by 158
  4. 2017CausalGAN: Learning Causal Implicit Generative Models with Adversarial Trainingcited by 152
  5. 2020Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effectscited by 72
  6. 2013Counterfactual Probabilities: Computational Methods, Bounds and Applicationscited by 67
  7. 2020A Framework For Contrastive Self-Supervised Learning And Designing A New Approachcited by 51
  8. 2018DeepMatch: Balancing Deep Covariate Representations for Causal Inference Using Adversarial Trainingcited by 42
  9. 2021Semiparametric counterfactual density estimationcited by 42
  10. 2018A Survey of Learning Causality with Data: Problems and Methodscited by 37
  11. 2019Adversarial Balancing-based Representation Learning for Causal Effect Inference with Observational Datacited by 29
  12. 2018Causal Identification under Markov Equivalencecited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 12; 2023: 22; 2024: 20; 2025: 22; 2026: 14.

97 papers.

Sort:

By area:

  1. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  2. 2022Diffusion Causal Models for Counterfactual Estimationcited by 87
  3. 2022Neural Causal Models for Counterfactual Identification and Estimationcited by 54
  4. 2023High Fidelity Image Counterfactuals with Probabilistic Causal Modelscited by 53
  5. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 43
  6. 2023Counterfactual Identifiability of Bijective Causal Modelscited by 42
  7. 2021Learning Neural Causal Models with Active Interventionscited by 40
  8. 2021Relating Graph Neural Networks to Structural Causal Modelscited by 39
  9. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  10. 2023Causal Interpretation of Self-Attention in Pre-Trained Transformerscited by 30
  11. 2023Sharp Bounds for Generalized Causal Sensitivity Analysiscited by 24
  12. 2022Causal Transportability for Visual Recognitioncited by 23
38 more
  1. 2023Passive learning of active causal strategies in agents and language modelscited by 21
  2. 2024Neural Causal Abstractionscited by 20
  3. 2024Counterfactual Image Editingcited by 20
  4. 2021VACA: Design of Variational Graph Autoencoders for Interventional and Counterfactual Queriescited by 18
  5. 2022Partial Identification of Treatment Effects with Implicit Generative Modelscited by 17
  6. 2023Counterfactual (Non-)identifiability of Learned Structural Causal Modelscited by 17
  7. 2023Towards Causal Foundation Model: on Duality between Causal Inference and Attentioncited by 17
  8. 2023A Neural Framework for Generalized Causal Sensitivity Analysiscited by 17
  9. 2023Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Modelcited by 14
  10. 2022Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxiescited by 13
  11. 2024Hybrid2 Neural ODE Causal Modeling and an Application to Glycemic Responsecited by 13
  12. 2025Black Box Causal Inference: Effect Estimation via Meta Predictioncited by 13
  13. 2022Unicorn: Reasoning about Configurable System Performance through the lens of Causalitycited by 12
  14. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 12
  15. 2024Mitigating attribute amplification in counterfactual image generationcited by 12
  16. 2022A Meta-Reinforcement Learning Algorithm for Causal Discoverycited by 11
  17. 2022Deep Causal Learning: Representation, Discovery and Inferencecited by 11
  18. 2024Modular Learning of Deep Causal Generative Models for High-dimensional Causal Inferencecited by 9
  19. 2024Learning Structural Causal Models through Deep Generative Models: Methods, Guarantees, and Challengescited by 8
  20. 2024Graph Neural Network Causal Explanation via Neural Causal Modelscited by 8
  21. 2025DeCaFlow: A deconfounding causal generative modelcited by 8
  22. 2025Partial Transportability for Domain Generalizationcited by 8
  23. 2024A Fixed-Point Approach for Causal Generative Modelingcited by 7
  24. 2022Finding and Listing Front-door Adjustment Setscited by 6
  25. 2023A Survey of Methods, Challenges and Perspectives in Causalitycited by 6
  26. 2023Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Modelscited by 6
  27. 2024Semi-Supervised Learning for Deep Causal Generative Modelscited by 6
  28. 2025Practical do-Shapley Explanations with Estimand-Agnostic Causal Inferencecited by 6
  29. 2024Causality from Bottom to Top: A Surveycited by 5
  30. 2021Causal Explanations of Structural Causal Modelscited by 4
  31. 2024Conditional Generative Models are Sufficient to Sample from Any Causal Effect Estimandcited by 4
  32. 2022The Causal Marginal Polytope for Bounding Treatment Effectscited by 3
  33. 2024Consistency of Neural Causal Partial Identificationcited by 3
  34. 2024chiSPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domainscited by 3
  35. 2024Estimating Causal Effects from Learned Causal Networkscited by 3
  36. 2024MACAW: A Causal Generative Model for Medical Imagingcited by 3
  37. 2025Spatial Deconfounder: Interference-Aware Deconfounding for Spatial Causal Inferencecited by 3
  38. 2023Towards Learning and Explaining Indirect Causal Effects in Neural Networkscited by 2

50 most cited of 97 All 97 (JSON)