# Optimizing NOTEARS Objectives via Topological Swaps

arXiv: https://arxiv.org/abs/2305.17277 · HTML: https://mlsanity.com/paper/2305.17277/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 56 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [CAM: Causal additive models, high-dimensional order search and penalized regression](https://mlsanity.com/paper/1310.1533/) (2013)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/1906.04477/) (2019)
- [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization](https://mlsanity.com/paper/2209.08037/) (2022)
- [High-dimensional learning of linear causal networks via inverse covariance estimation](https://mlsanity.com/paper/1311.3492/) (2013)
- [Permutation-Based Causal Structure Learning with Unknown Intervention Targets](https://mlsanity.com/paper/1910.09007/) (2019)
- [DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks](https://mlsanity.com/paper/2010.09133/) (2020)
- [On Causal Discovery with Equal Variance Assumption](https://mlsanity.com/paper/1807.03419/) (2018)
- [Ordering-Based Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/2105.06631/) (2021)
- [Learning linear structural equation models in polynomial time and sample complexity](https://mlsanity.com/paper/1707.04673/) (2017)
- [High-dimensional consistency in score-based and hybrid structure learning](https://mlsanity.com/paper/1507.02608/) (2015)
- [Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions](https://mlsanity.com/paper/1301.6697/) (2013)
- [On the Convergence of Continuous Constrained Optimization for Structure Learning](https://mlsanity.com/paper/2011.11150/) (2020)
- [Concave Penalized Estimation of Sparse Gaussian Bayesian Networks](https://mlsanity.com/paper/1401.0852/) (2014)
- [Learning Identifiable Gaussian Bayesian Networks in Polynomial Time and Sample Complexity](https://mlsanity.com/paper/1703.01196/) (2017)
- [Bayesian Network Learning via Topological Order](https://mlsanity.com/paper/1701.05654/) (2017)
- [Optimizing regularized Cholesky score for order-based learning of Bayesian networks](https://mlsanity.com/paper/1904.12360/) (2019)
- [A Bregman Method for Structure Learning on Sparse Directed Acyclic Graphs](https://mlsanity.com/paper/2011.02764/) (2020)

## Built on by (most cited first, 17 of 17)

- [Stable Differentiable Causal Discovery](https://mlsanity.com/paper/2311.10263/) (2023)
- [CoLiDE: Concomitant Linear DAG Estimation](https://mlsanity.com/paper/2310.02895/) (2023)
- [Ordering-Based Causal Discovery for Linear and Nonlinear Relations](https://mlsanity.com/paper/2410.05890/) (2024)
- [Constraint-Free Structure Learning with Smooth Acyclic Orientations](https://mlsanity.com/paper/2309.08406/) (2023)
- [C2P: Featuring Large Language Models with Causal Reasoning](https://mlsanity.com/paper/2407.18069/) (2024)
- [Optimal Transport for Structure Learning Under Missing Data](https://mlsanity.com/paper/2402.15255/) (2024)
- [Scalable and Flexible Causal Discovery with an Efficient Test for Adjacency](https://mlsanity.com/paper/2406.09177/) (2024)
- [ProDAG: Projected Variational Inference for Directed Acyclic Graphs](https://mlsanity.com/paper/2405.15167/) (2024)
- [Score-based Greedy Search for Structure Identification of Partially Observed Linear Causal Models](https://mlsanity.com/paper/2510.04378/) (2025)
- [Concomitant DAG Learning: On the Roles of Noise Adaptivity, Sparsity, and Non-negativity](https://mlsanity.com/paper/2605.23537/) (2026)
- [Discovering Dynamic Effective Connectome of Brain with Bayesian Dynamic DAG Learning](https://mlsanity.com/paper/2309.07080/) (2023)
- [Interventional Causal Structure Discovery over Graphical Models with Convergence and Optimality Guarantees](https://mlsanity.com/paper/2408.04819/) (2024)
- [Causal Order Discovery based on Monotonic SCMs](https://mlsanity.com/paper/2410.19870/) (2024)
- [Differentiable Structure Learning and Causal Discovery for General Binary Data](https://mlsanity.com/paper/2509.21658/) (2025)
- [SC3D: Dynamic and Differentiable Causal Discovery for Temporal and Instantaneous Graphs](https://mlsanity.com/paper/2602.02830/) (2026)
- [Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives](https://mlsanity.com/paper/2604.02250/) (2026)
- [Structure Learning on Clustered Data](https://mlsanity.com/paper/2607.08238/) (2026)

## Abstract (arXiv)

Recently, an intriguing class of non-convex optimization problems has emerged in the context of learning directed acyclic graphs (DAGs). These problems involve minimizing a given loss or score function, subject to a non-convex continuous constraint that penalizes the presence of cycles in a graph. In this work, we delve into the optimization challenges associated with this class of non-convex programs. To address these challenges, we propose a bi-level algorithm that leverages the non-convex constraint in a novel way. The outer level of the algorithm optimizes over topological orders by iteratively swapping pairs of nodes within the topological order of a DAG. A key innovation of our approach is the development of an effective method for generating a set of candidate swapping pairs for each iteration. At the inner level, given a topological order, we utilize off-the-shelf solvers that can handle linear constraints. The key advantage of our proposed algorithm is that it is guaranteed to find a local minimum or a KKT point under weaker conditions compared to previous work and finds solutions with lower scores. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches in terms of achieving a better score. Additionally, our method can also be used as a post-processing algorithm to significantly improve the score of other algorithms. Code implementing the proposed method is available at https://github.com/duntrain/topo.