# Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Data

arXiv: https://arxiv.org/abs/2307.03410 · HTML: https://mlsanity.com/paper/2307.03410/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 17 of 17)

- [Spectral Methods for Data Science: A Statistical Perspective](https://mlsanity.com/paper/2012.08496/) (2020)
- [On the Global Linear Convergence of Frank-Wolfe Optimization Variants](https://mlsanity.com/paper/1511.05932/) (2015)
- [Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares](https://mlsanity.com/paper/1411.0347/) (2014)
- [EDGAR-CORPUS: Billions of Tokens Make The World Go Round](https://mlsanity.com/paper/2109.14394/) (2021)
- [Distributed Coordinate Descent Method for Learning with Big Data](https://mlsanity.com/paper/1310.2059/) (2013)
- [Revisiting Frank-Wolfe for Polytopes: Strict Complementarity and Sparsity](https://mlsanity.com/paper/2006.00558/) (2020)
- [Spectral Frank-Wolfe Algorithm: Strict Complementarity and Linear Convergence](https://mlsanity.com/paper/2006.01719/) (2020)
- [A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning](https://mlsanity.com/paper/1404.2644/) (2014)
- [DUAL-LOCO: Distributing Statistical Estimation Using Random Projections](https://mlsanity.com/paper/1506.02554/) (2015)
- [kFW: A Frank-Wolfe style algorithm with stronger subproblem oracles](https://mlsanity.com/paper/2006.16142/) (2020)
- [Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Data](https://mlsanity.com/paper/1610.03045/) (2016)
- [Fast Distributed Coordinate Descent for Non-Strongly Convex Losses](https://mlsanity.com/paper/1405.5300/) (2014)
- [A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm](https://mlsanity.com/paper/1712.07495/) (2017)
- [DECOrrelated feature space partitioning for distributed sparse regression](https://mlsanity.com/paper/1602.02575/) (2016)
- [Model selection for high-dimensional linear regression with dependent observations](https://mlsanity.com/paper/1906.07395/) (2019)
- [Communication-Efficient Asynchronous Stochastic Frank-Wolfe over Nuclear-norm Balls](https://mlsanity.com/paper/1910.07703/) (2019)
- [Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data](https://mlsanity.com/paper/2103.14626/) (2021)

## Abstract (arXiv)

Feature-distributed data, referred to data partitioned by features and stored across multiple computing nodes, are increasingly common in applications with a large number of features. This paper proposes a two-stage relaxed greedy algorithm (TSRGA) for applying multivariate linear regression to such data. The main advantage of TSRGA is that its communication complexity does not depend on the feature dimension, making it highly scalable to very large data sets. In addition, for multivariate response variables, TSRGA can be used to yield low-rank coefficient estimates. The fast convergence of TSRGA is validated by simulation experiments. Finally, we apply the proposed TSRGA in a financial application that leverages unstructured data from the 10-K reports, demonstrating its usefulness in applications with many dense large-dimensional matrices.