# Revisiting Differentiable Structure Learning: Inconsistency of ℓ1 Penalty and Beyond

arXiv: https://arxiv.org/abs/2410.18396 · HTML: https://mlsanity.com/paper/2410.18396/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [DYNOTEARS: Structure Learning from Time-Series Data](https://mlsanity.com/paper/2002.00498/) (2020)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization](https://mlsanity.com/paper/2209.08037/) (2022)
- [Masked Gradient-Based Causal Structure Learning](https://mlsanity.com/paper/1910.08527/) (2019)
- [High-dimensional learning of linear causal networks via inverse covariance estimation](https://mlsanity.com/paper/1311.3492/) (2013)
- [DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks](https://mlsanity.com/paper/2010.09133/) (2020)
- [Differentiable Causal Discovery Under Unmeasured Confounding](https://mlsanity.com/paper/2010.06978/) (2020)
- [Structural Agnostic Modeling: Adversarial Learning of Causal Graphs](https://mlsanity.com/paper/1803.04929/) (2018)
- [Unsuitability of NOTEARS for Causal Graph Discovery](https://mlsanity.com/paper/2104.05441/) (2021)
- [On the Convergence of Continuous Constrained Optimization for Structure Learning](https://mlsanity.com/paper/2011.11150/) (2020)
- [Structure Learning with Continuous Optimization: A Sober Look and Beyond](https://mlsanity.com/paper/2304.02146/) (2023)
- [Towards Federated Bayesian Network Structure Learning with Continuous Optimization](https://mlsanity.com/paper/2110.09356/) (2021)
- [Stable Differentiable Causal Discovery](https://mlsanity.com/paper/2311.10263/) (2023)
- [Causal Discovery from Incomplete Data: A Deep Learning Approach](https://mlsanity.com/paper/2001.05343/) (2020)
- [Truncated Matrix Power Iteration for Differentiable DAG Learning](https://mlsanity.com/paper/2208.14571/) (2022)
- [MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models](https://mlsanity.com/paper/2205.13869/) (2022)
- [Differentiable Causal Discovery Under Latent Interventions](https://mlsanity.com/paper/2203.02336/) (2022)
- [Learning causal representations for robust domain adaptation](https://mlsanity.com/paper/2011.06317/) (2020)
- [FedDAG: Federated DAG Structure Learning](https://mlsanity.com/paper/2112.03555/) (2021)
- [CoLiDE: Concomitant Linear DAG Estimation](https://mlsanity.com/paper/2310.02895/) (2023)
- [Multi-task Learning of Order-Consistent Causal Graphs](https://mlsanity.com/paper/2111.02545/) (2021)
- [Global Optimality in Bivariate Gradient-based DAG Learning](https://mlsanity.com/paper/2306.17378/) (2023)
- [Deconfounded Score Method: Scoring DAGs with Dense Unobserved Confounding](https://mlsanity.com/paper/2103.15106/) (2021)

## Built on by (most cited first, 1 of 1)

- [Causal-Copilot: An Autonomous Causal Analysis Agent](https://mlsanity.com/paper/2504.13263/) (2025)

## Abstract (arXiv)

Recent advances in differentiable structure learning have framed the combinatorial problem of learning directed acyclic graphs as a continuous optimization problem. Various aspects, including data standardization, have been studied to identify factors that influence the empirical performance of these methods. In this work, we investigate critical limitations in differentiable structure learning methods, focusing on settings where the true structure can be identified up to Markov equivalence classes, particularly in the linear Gaussian case. While Ng et al. (2024) highlighted potential non-convexity issues in this setting, we demonstrate and explain why the use of ℓ1-penalized likelihood in such cases is fundamentally inconsistent, even if the global optimum of the optimization problem can be found. To resolve this limitation, we develop a hybrid differentiable structure learning method based on ℓ0-penalized likelihood with hard acyclicity constraint, where the ℓ0 penalty can be approximated by different techniques including Gumbel-Softmax. Specifically, we first estimate the underlying moral graph, and use it to restrict the search space of the optimization problem, which helps alleviate the non-convexity issue. Experimental results show that the proposed method enhances empirical performance both before and after data standardization, providing a more reliable path for future advancements in differentiable structure learning, especially for learning Markov equivalence classes.