mlsanityData to 14 Oct 2026

Straight-Through meets Sparse Recovery: the Support Exploration Algorithm

arXiv 2301.13584 ↗First posted 31 Jan 2023cs.LG cs.AI math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

The it straight-through estimator (STE) is commonly used to optimize quantized neural networks, yet its contexts of effective performance are still unclear despite empirical successes.To make a step forward in this comprehension, we apply STE to a well-understood problem: it sparse support recovery. We introduce the it Support Exploration Algorithm (SEA), a novel algorithm promoting sparsity, and we analyze its performance in support recovery (a.k.a. model selection) problems. SEA explores more supports than the state-of-the-art, leading to superior performance in experiments, especially when the columns of A are strongly coherent.The theoretical analysis considers recovery guarantees when the linear measurements matrix A satisfies the it Restricted Isometry Property (RIP).The sufficient conditions of recovery are comparable but more stringent than those of the state-of-the-art in sparse support recovery. Their significance lies mainly in their applicability to an instance of the STE.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 13 of its 48 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

13 of 48 references.

References · 13

  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  2. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  3. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  4. 2017Training Quantized Nets: A Deeper Understandingcited by 75
  5. 2019Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimizationcited by 60
  6. 2021A comprehensive review of Binary Neural Networkcited by 45
  7. 2013Exact Support Recovery for Sparse Spikes Deconvolutioncited by 22
  8. 2017The High-Dimensional Geometry of Binary Neural Networkscited by 22
  9. 2019Geometry and Symmetry in Short-and-Sparse Deconvolutioncited by 11
  10. 2020Sparse Convex Optimization via Adaptively Regularized Hard Thresholdingcited by 10
  11. 2020Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classificationcited by 8
  12. 2019Straight-Through Estimator as Projected Wasserstein Gradient Flowcited by 7
1 more
  1. 2017Deconvolution of Point Sources: A Sampling Theorem and Robustness Guaranteescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimatorcited by 4
  2. 2025Beyond Discreteness: Sample Complexity Analysis of Straight-Through Estimator for 1-bit Quantizationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2021Support Recovery in Universal One-bit Compressed Sensing
  2. 2022Improved Support Recovery in Universal One-bit Compressed Sensing
  3. 2024Robust Training of Neural Networks at Arbitrary Precision and Sparsity
  4. 2024Uniform Recovery Guarantees for Quantized Corrupted Sensing Using Structured or Generative Priors
  5. 2021Quantized Proximal Averaging Network for Analysis Sparse Coding
  6. 2021Support Recovery of Sparse Signals from a Mixture of Linear Measurements
  7. 2026Towards a mathematical theory of superposition
  8. 2025A Recovery Theory for Diffusion Priors: Deterministic Analysis of the Implicit Prior Algorithm