mlsanityData to 14 Oct 2026

Structure Learning with Continuous Optimization: A Sober Look and Beyond

arXiv 2304.02146 ↗First posted 4 Apr 2023cs.LG stat.MLcited by 36

Not on the map. Why?

Abstract (arXiv)

This paper investigates in which cases continuous optimization for directed acyclic graph (DAG) structure learning can and cannot perform well and why this happens, and suggests possible directions to make the search procedure more reliable. Reisach et al. (2021) suggested that the remarkable performance of several continuous structure learning approaches is primarily driven by a high agreement between the order of increasing marginal variances and the topological order, and demonstrated that these approaches do not perform well after data standardization. We analyze this phenomenon for continuous approaches assuming equal and non-equal noise variances, and show that the statement may not hold in either case by providing counterexamples, justifications, and possible alternative explanations. We further demonstrate that nonconvexity may be a main concern especially for the non-equal noise variances formulation, while recent advances in continuous structure learning fail to achieve improvement in this case. Our findings suggest that future works should take into account the non-equal noise variances formulation to handle more general settings and for a more comprehensive empirical evaluation. Lastly, we provide insights into other aspects of the search procedure, including thresholding and sparsity, and show that they play an important role in the final solutions.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 35 of its 84 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

35 of 84 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
  3. 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 331
  4. 2019Gradient-Based Neural DAG Learningcited by 227
  5. 2020Differentiable Causal Discovery from Interventional Datacited by 218
  6. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  7. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 211
  8. 2019Learning Sparse Nonparametric DAGscited by 207
  9. 2020DYNOTEARS: Structure Learning from Time-Series Datacited by 200
  10. 2020On the Role of Sparsity and DAG Constraints for Learning Linear DAGscited by 191
  11. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 175
  12. 2021Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Gamecited by 161
23 more
  1. 2022DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterizationcited by 140
  2. 2019Masked Gradient-Based Causal Structure Learningcited by 89
  3. 2013High-dimensional learning of linear causal networks via inverse covariance estimationcited by 87
  4. 2020DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networkscited by 66
  5. 2020Differentiable Causal Discovery Under Unmeasured Confoundingcited by 56
  6. 2021DAGs with No Curl: An Efficient DAG Structure Learning Approachcited by 53
  7. 2018Structural Agnostic Modeling: Adversarial Learning of Causal Graphscited by 48
  8. 2021Unsuitability of NOTEARS for Causal Graph Discoverycited by 47
  9. 2022Differentiable DAG Samplingcited by 47
  10. 2020On the Convergence of Continuous Constrained Optimization for Structure Learningcited by 38
  11. 2021Towards Federated Bayesian Network Structure Learning with Continuous Optimizationcited by 31
  12. 2020Causal Discovery from Incomplete Data: A Deep Learning Approachcited by 26
  13. 2014Support recovery without incoherence: A case for nonconvex regularizationcited by 25
  14. 2022Truncated Matrix Power Iteration for Differentiable DAG Learningcited by 23
  15. 2022MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Modelscited by 22
  16. 2022Differentiable Causal Discovery Under Latent Interventionscited by 21
  17. 2020Learning causal representations for robust domain adaptationcited by 18
  18. 2021FedDAG: Federated DAG Structure Learningcited by 17
  19. 2020Causal Discovery with Multi-Domain LiNGAM for Latent Factorscited by 13
  20. 2021Multi-task Learning of Order-Consistent Causal Graphscited by 10
  21. 2021Deconfounded Score Method: Scoring DAGs with Dense Unobserved Confoundingcited by 9
  22. 2022NeurIPS Competition Instructions and Guide: Causal Insights for Learning Paths in Educationcited by 4
  23. 2022On the Sparse DAG Structure Learning Based on Adaptive Lassocited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 14; 2025: 8; 2026: 12.

36 papers.

Sort:

By area:

  1. 2023Stable Differentiable Causal Discoverycited by 31
  2. 2024Sample, estimate, aggregate: A recipe for causal discovery foundation modelscited by 21
  3. 2024Federated Causal Discovery from Heterogeneous Datacited by 15
  4. 2023Constraint-Free Structure Learning with Smooth Acyclic Orientationscited by 10
  5. 2025Unitless Unrestricted Markov-Consistent SCM Generation: Better Benchmark Datasets for Causal Discoverycited by 8
  6. 2024Markov Equivalence and Consistency in Differentiable Structure Learningcited by 7
  7. 2024A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discoverycited by 7
  8. 2024Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Methodcited by 6
  9. 2025Analytic DAG Constraints for Differentiable DAG Learningcited by 5
  10. 2024AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUscited by 4
  11. 2025Embracing Discrete Search: A Reasonable Approach to Causal Structure Learningcited by 4
  12. 2025The Robustness of Differentiable Causal Discovery in Misspecified Scenarioscited by 4
24 more
  1. 2024Simulation-based Benchmarking for Causal Structure Learning in Gene Perturbation Experimentscited by 3
  2. 2024Sortability of Time Series Datacited by 3
  3. 2024Differentiable Causal Discovery For Latent Hierarchical Causal Modelscited by 3
  4. 2026Score-Based Causal Discovery of Latent Variable Causal Modelscited by 3
  5. 2024Score-matching-based Structure Learning for Temporal Data on Networkscited by 2
  6. 2025Robust Causal Discovery under Imperfect Structural Constraintscited by 2
  7. 2024Interventional Causal Structure Discovery over Graphical Models with Convergence and Optimality Guaranteescited by 1
  8. 2024Revisiting Differentiable Structure Learning: Inconsistency of ℓ1 Penalty and Beyondcited by 1
  9. 2024Causal Order Discovery based on Monotonic SCMscited by 1
  10. 2024Fast Causal Discovery by Approximate Kernel-based Generalized Score Functions with Linear Computational Complexitycited by 1
  11. 2025Less Greedy Equivalence Searchcited by 1
  12. 2025Beyond Pairwise Connections: Extracting High-Order Functional Brain Network Structures under Global Constraintscited by 1
  13. 2025Differentially Private and Federated Structure Learning in Bayesian Networkscited by 1
  14. 2026Transformer Is Inherently a Causal Learnercited by 1
  15. 2026Learning Causal Structure of Time Series using Best Order Score Searchcited by 0
  16. 2026Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectivescited by 0
  17. 2026Causal Discovery as Dialectical Aggregation: A Quantitative Argumentation Frameworkcited by 0
  18. 2026A Topological Sorting Criterion for Random Causal Directed Acyclic Graphscited by 0
  19. 2026Support Selection Beyond Smooth DAG Exactness: Completion Geometry,Score Margins, and Selective Certificatescited by 0
  20. 2026Replicating TRACE: A Practitioner's Guide to Its Threshold and Particle Budgetcited by 0
  21. 2026ITSY: Causal Discovery From Irregular Time-Series Datacited by 0
  22. 2026Nonnegative DAG Learning via Concomitant Estimationcited by 0
  23. 2026Directional Evidence Guided Search-Space Reduction for Exact DAG Learningcited by 0
  24. 2026Learning Non-Gaussian SEMs with Conditional Risk, and Why the Direction of Recursion Matterscited by 0