# Structure Learning with Continuous Optimization: A Sober Look and Beyond

arXiv: https://arxiv.org/abs/2304.02146 · HTML: https://mlsanity.com/paper/2304.02146/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 84 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [D'ya like DAGs? A Survey on Structure Learning and Causal Discovery](https://mlsanity.com/paper/2103.02582/) (2021)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [DYNOTEARS: Structure Learning from Time-Series Data](https://mlsanity.com/paper/2002.00498/) (2020)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization](https://mlsanity.com/paper/2209.08037/) (2022)
- [Masked Gradient-Based Causal Structure Learning](https://mlsanity.com/paper/1910.08527/) (2019)
- [High-dimensional learning of linear causal networks via inverse covariance estimation](https://mlsanity.com/paper/1311.3492/) (2013)
- [DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks](https://mlsanity.com/paper/2010.09133/) (2020)
- [Differentiable Causal Discovery Under Unmeasured Confounding](https://mlsanity.com/paper/2010.06978/) (2020)
- [DAGs with No Curl: An Efficient DAG Structure Learning Approach](https://mlsanity.com/paper/2106.07197/) (2021)
- [Structural Agnostic Modeling: Adversarial Learning of Causal Graphs](https://mlsanity.com/paper/1803.04929/) (2018)
- [Unsuitability of NOTEARS for Causal Graph Discovery](https://mlsanity.com/paper/2104.05441/) (2021)
- [Differentiable DAG Sampling](https://mlsanity.com/paper/2203.08509/) (2022)
- [On the Convergence of Continuous Constrained Optimization for Structure Learning](https://mlsanity.com/paper/2011.11150/) (2020)
- [Towards Federated Bayesian Network Structure Learning with Continuous Optimization](https://mlsanity.com/paper/2110.09356/) (2021)
- [Causal Discovery from Incomplete Data: A Deep Learning Approach](https://mlsanity.com/paper/2001.05343/) (2020)
- [Support recovery without incoherence: A case for nonconvex regularization](https://mlsanity.com/paper/1412.5632/) (2014)
- [Truncated Matrix Power Iteration for Differentiable DAG Learning](https://mlsanity.com/paper/2208.14571/) (2022)
- [MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models](https://mlsanity.com/paper/2205.13869/) (2022)
- [Differentiable Causal Discovery Under Latent Interventions](https://mlsanity.com/paper/2203.02336/) (2022)
- [Learning causal representations for robust domain adaptation](https://mlsanity.com/paper/2011.06317/) (2020)
- [FedDAG: Federated DAG Structure Learning](https://mlsanity.com/paper/2112.03555/) (2021)

## Built on by (most cited first, 30 of 36)

- [Stable Differentiable Causal Discovery](https://mlsanity.com/paper/2311.10263/) (2023)
- [Sample, estimate, aggregate: A recipe for causal discovery foundation models](https://mlsanity.com/paper/2402.01929/) (2024)
- [Federated Causal Discovery from Heterogeneous Data](https://mlsanity.com/paper/2402.13241/) (2024)
- [Constraint-Free Structure Learning with Smooth Acyclic Orientations](https://mlsanity.com/paper/2309.08406/) (2023)
- [Unitless Unrestricted Markov-Consistent SCM Generation: Better Benchmark Datasets for Causal Discovery](https://mlsanity.com/paper/2503.17037/) (2025)
- [Markov Equivalence and Consistency in Differentiable Structure Learning](https://mlsanity.com/paper/2410.06163/) (2024)
- [A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discovery](https://mlsanity.com/paper/2410.06407/) (2024)
- [Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Method](https://mlsanity.com/paper/2405.13100/) (2024)
- [Analytic DAG Constraints for Differentiable DAG Learning](https://mlsanity.com/paper/2503.19218/) (2025)
- [AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs](https://mlsanity.com/paper/2403.03772/) (2024)
- [Embracing Discrete Search: A Reasonable Approach to Causal Structure Learning](https://mlsanity.com/paper/2510.04970/) (2025)
- [The Robustness of Differentiable Causal Discovery in Misspecified Scenarios](https://mlsanity.com/paper/2510.12503/) (2025)
- [Simulation-based Benchmarking for Causal Structure Learning in Gene Perturbation Experiments](https://mlsanity.com/paper/2407.06015/) (2024)
- [Sortability of Time Series Data](https://mlsanity.com/paper/2407.13313/) (2024)
- [Differentiable Causal Discovery For Latent Hierarchical Causal Models](https://mlsanity.com/paper/2411.19556/) (2024)
- [Score-Based Causal Discovery of Latent Variable Causal Models](https://mlsanity.com/paper/2605.20396/) (2026)
- [Score-matching-based Structure Learning for Temporal Data on Networks](https://mlsanity.com/paper/2412.07469/) (2024)
- [Robust Causal Discovery under Imperfect Structural Constraints](https://mlsanity.com/paper/2511.06790/) (2025)
- [Interventional Causal Structure Discovery over Graphical Models with Convergence and Optimality Guarantees](https://mlsanity.com/paper/2408.04819/) (2024)
- [Revisiting Differentiable Structure Learning: Inconsistency of ℓ1 Penalty and Beyond](https://mlsanity.com/paper/2410.18396/) (2024)
- [Causal Order Discovery based on Monotonic SCMs](https://mlsanity.com/paper/2410.19870/) (2024)
- [Fast Causal Discovery by Approximate Kernel-based Generalized Score Functions with Linear Computational Complexity](https://mlsanity.com/paper/2412.17717/) (2024)
- [Less Greedy Equivalence Search](https://mlsanity.com/paper/2506.22331/) (2025)
- [Beyond Pairwise Connections: Extracting High-Order Functional Brain Network Structures under Global Constraints](https://mlsanity.com/paper/2510.09175/) (2025)
- [Differentially Private and Federated Structure Learning in Bayesian Networks](https://mlsanity.com/paper/2512.01708/) (2025)
- [Transformer Is Inherently a Causal Learner](https://mlsanity.com/paper/2601.05647/) (2026)
- [Learning Causal Structure of Time Series using Best Order Score Search](https://mlsanity.com/paper/2603.05370/) (2026)
- [Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives](https://mlsanity.com/paper/2604.02250/) (2026)
- [Causal Discovery as Dialectical Aggregation: A Quantitative Argumentation Framework](https://mlsanity.com/paper/2604.23633/) (2026)
- [A Topological Sorting Criterion for Random Causal Directed Acyclic Graphs](https://mlsanity.com/paper/2605.06288/) (2026)

## Abstract (arXiv)

This paper investigates in which cases continuous optimization for directed acyclic graph (DAG) structure learning can and cannot perform well and why this happens, and suggests possible directions to make the search procedure more reliable. Reisach et al. (2021) suggested that the remarkable performance of several continuous structure learning approaches is primarily driven by a high agreement between the order of increasing marginal variances and the topological order, and demonstrated that these approaches do not perform well after data standardization. We analyze this phenomenon for continuous approaches assuming equal and non-equal noise variances, and show that the statement may not hold in either case by providing counterexamples, justifications, and possible alternative explanations. We further demonstrate that nonconvexity may be a main concern especially for the non-equal noise variances formulation, while recent advances in continuous structure learning fail to achieve improvement in this case. Our findings suggest that future works should take into account the non-equal noise variances formulation to handle more general settings and for a more comprehensive empirical evaluation. Lastly, we provide insights into other aspects of the search procedure, including thresholding and sparsity, and show that they play an important role in the final solutions.