# Useful Compact Representations for Data-Fitting

arXiv: https://arxiv.org/abs/2403.12206 · HTML: https://mlsanity.com/paper/2403.12206/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 10 of 10)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning](https://mlsanity.com/paper/1712.06559/) (2017)
- [A Stochastic Quasi-Newton Method for Large-Scale Optimization](https://mlsanity.com/paper/1401.7020/) (2014)
- [Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties](https://mlsanity.com/paper/1403.3862/) (2014)
- [Regularization of Limited Memory Quasi-Newton Methods for Large-Scale Nonconvex Minimization](https://mlsanity.com/paper/1911.04584/) (2019)
- [Algorithm XXX: SC-SR1: Matlab software for solving shape-changing L-SR1 trust-region subproblems](https://mlsanity.com/paper/1607.03533/) (2016)
- [Large-scale Optimization with Linear Equality Constraints using Reduced Compact Representation](https://mlsanity.com/paper/2101.11048/) (2021)
- [Compact representations of structured BFGS matrices](https://mlsanity.com/paper/2208.00057/) (2022)
- [Shape-Changing Trust-Region Methods Using Multipoint Symmetric Secant Matrices](https://mlsanity.com/paper/2209.12057/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Nonmonotone Gradient-Based Algorithm for Symmetric Nonnegative Matrix Factorization and Graph Clustering](https://mlsanity.com/paper/2606.02887/) (2026)

## Abstract (arXiv)

For minimization problems without 2nd derivative information, methods that estimate Hessian matrices can be very effective. However, conventional techniques generate dense matrices that are prohibitive for large problems. Limited-memory compact representations express the dense arrays in terms of a low rank representation and have become the state-of-the-art for software implementations on large deterministic problems. We develop new compact representations that are parameterized by a choice of vectors and that reduce to existing well known formulas for special choices. We demonstrate effectiveness of the compact representations for large eigenvalue computations, tensor factorizations and nonlinear regressions.