# Flexible Multi-layer Sparse Approximations of Matrices and Applications

arXiv: https://arxiv.org/abs/1506.07300 · HTML: https://mlsanity.com/paper/1506.07300/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2015.

## Built on by (most cited first, 5 of 5)

- [Monarch: Expressive Structured Matrices for Efficient and Accurate Training](https://mlsanity.com/paper/2204.00595/) (2022)
- [Two Sparse Matrices are Better than One: Sparsifying Neural Networks with Double Sparse Factorization](https://mlsanity.com/paper/2409.18850/) (2024)
- [Efficient Identification of Butterfly Sparse Matrix Factorizations](https://mlsanity.com/paper/2110.01230/) (2021)
- [Learning distributed representations with efficient SoftMax normalization](https://mlsanity.com/paper/2303.17475/) (2023)
- [Identifiability in Two-Layer Sparse Matrix Factorization](https://mlsanity.com/paper/2110.01235/) (2021)

## Abstract (arXiv)

The computational cost of many signal processing and machine learning techniques is often dominated by the cost of applying certain linear operators to high-dimensional vectors. This paper introduces an algorithm aimed at reducing the complexity of applying linear operators in high dimension by approximately factorizing the corresponding matrix into few sparse factors. The approach relies on recent advances in non-convex optimization. It is first explained and analyzed in details and then demonstrated experimentally on various problems including dictionary learning for image denoising, and the approximation of large matrices arising in inverse problems.