# Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation

arXiv: https://arxiv.org/abs/1710.10769 · HTML: https://mlsanity.com/paper/1710.10769/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2017.

## Built on by (most cited first, 4 of 4)

- [Diagonal Nonlinear Transformations Preserve Structure in Covariance and Precision Matrices](https://mlsanity.com/paper/2107.04136/) (2021)
- [Learning Interpretable Models for Coupled Networks Under Domain Constraints](https://mlsanity.com/paper/2104.09069/) (2021)
- [Learning Massive-scale Partial Correlation Networks in Clinical Multi-omics Studies with HP-ACCORD](https://mlsanity.com/paper/2412.11554/) (2024)
- [The Ubiquitous Sparse Matrix-Matrix Products](https://mlsanity.com/paper/2508.04077/) (2025)

## Abstract (arXiv)

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ~819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.