# Sharp Convergence Rates for Matching Pursuit

arXiv: https://arxiv.org/abs/2307.07679 · HTML: https://mlsanity.com/paper/2307.07679/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 33 references that are in the map.
- First posted on arXiv 15 Jul 2023.

## Builds on (most cited first, 5 of 5)

- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Characterization of the Variation Spaces Corresponding to Shallow Neural Networks](https://mlsanity.com/paper/2106.15002/) (2021)

## Built on by (most cited first, 2 of 2)

- [Simplicity is Key: An Unsupervised Pretraining Approach for Sparse Radio Channels](https://mlsanity.com/paper/2505.13055/) (2025)
- [Analyze Additive and Interaction Effects via Collaborative Trees](https://mlsanity.com/paper/2405.11477/) (2024)

## Abstract (arXiv)

We study the fundamental limits of matching pursuit, or the pure greedy algorithm, for approximating a target function  f  by a linear combination fn of n elements from a dictionary. When the target function is contained in the variation space corresponding to the dictionary, many impressive works over the past few decades have obtained upper and lower bounds on the error |f-fn| of matching pursuit, but they do not match. The main contribution of this paper is to close this gap and obtain a sharp characterization of the decay rate, n-α, of matching pursuit. Specifically, we construct a worst case dictionary which shows that the existing best upper bound cannot be significantly improved. It turns out that, unlike other greedy algorithm variants which converge at the optimal rate  n-1/2, the convergence rate n-α is suboptimal. Here, α ≈ 0.182 is determined by the solution to a certain non-linear equation.