# Efficient Network Automatic Relevance Determination

arXiv: https://arxiv.org/abs/2506.12352 · HTML: https://mlsanity.com/paper/2506.12352/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 54 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Builds on (most cited first, 9 of 9)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [The Graphical Horseshoe Estimator for Inverse Covariance Matrices](https://mlsanity.com/paper/1707.06661/) (2017)
- [Efficient Multitask Feature and Relationship Learning](https://mlsanity.com/paper/1702.04423/) (2017)
- [A GAMP Based Low Complexity Sparse Bayesian Learning Algorithm](https://mlsanity.com/paper/1703.03044/) (2017)
- [Sparse Bayesian Learning via Stepwise Regression](https://mlsanity.com/paper/2106.06095/) (2021)
- [Asymptotically Normal and Efficient Estimation of Covariate-Adjusted Gaussian Graphical Model](https://mlsanity.com/paper/1309.5923/) (2013)
- [Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso](https://mlsanity.com/paper/1708.08911/) (2017)
- [Bayesian Structure Learning in Multi-layered Genomic Networks](https://mlsanity.com/paper/2002.07122/) (2020)
- [A generalized likelihood based Bayesian approach for scalable joint regression and covariance selection in high dimensions](https://mlsanity.com/paper/2201.05653/) (2022)

## Built on by (most cited first, 2 of 2)

- [Joint Model and Data Sparsification via the Marginal Likelihood](https://mlsanity.com/paper/2605.29908/) (2026)
- [A Weighted Kernel Method for Approximation that Adapts to Learned Multivariable Structure](https://mlsanity.com/paper/2609.16606/) (2026)

## Abstract (arXiv)

We propose Network Automatic Relevance Determination (NARD), an extension of ARD for linearly probabilistic models, to simultaneously model sparse relationships between inputs X ∈ mathbb Rd × N and outputs Y ∈ mathbb Rm × N, while capturing the correlation structure among the Y. NARD employs a matrix normal prior which contains a sparsity-inducing parameter to identify and discard irrelevant features, thereby promoting sparsity in the model. Algorithmically, it iteratively updates both the precision matrix and the relationship between Y and the refined inputs. To mitigate the computational inefficiencies of the mathcal O(m3 + d3) cost per iteration, we introduce Sequential NARD, which evaluates features sequentially, and a Surrogate Function Method, leveraging an efficient approximation of the marginal likelihood and simplifying the calculation of determinant and inverse of an intermediate matrix. Combining the Sequential update with the Surrogate Function method further reduces computational costs. The computational complexity per iteration for these three methods is reduced to mathcal O(m3+p3), mathcal O(m3 + d2), mathcal O(m3+p2), respectively, where p ll d is the final number of features in the model. Our methods demonstrate significant improvements in computational efficiency with comparable performance on both synthetic and real-world datasets.