mlsanityData to 14 Oct 2026

Unveiling Hidden Convexity in Deep Learning: a Sparse Signal Processing Perspective

arXiv 2603.23831 ↗First posted 25 Mar 2026cs.LG eess.SP stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs), particularly those using Rectified Linear Unit (ReLU) activation functions, have achieved remarkable success across diverse machine learning tasks, including image recognition, audio processing, and language modeling. Despite this success, the non-convex nature of DNN loss functions complicates optimization and limits theoretical understanding. In this paper, we highlight how recently developed convex equivalences of ReLU NNs and their connections to sparse signal processing models can address the challenges of training and understanding NNs. Recent research has uncovered several hidden convexities in the loss landscapes of certain NN architectures, notably two-layer ReLU networks and other deeper or varied architectures. This paper seeks to provide an accessible and educational overview that bridges recent advances in the mathematics of deep learning with traditional signal processing, encouraging broader signal processing applications.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

20 of 46 references.

References · 20

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  3. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  4. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 98
  5. 2020The Global Landscape of Neural Networks: An Overviewcited by 57
  6. 2022Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositionscited by 31
  7. 2023Training Neural Networks is NP-Hard in Fixed Dimensioncited by 23
  8. 2023Deep Learning Meets Sparse Regularization: A Signal Processing Perspectivecited by 21
  9. 2019Convex Formulation of Overparameterized Deep Neural Networkscited by 16
  10. 2021Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalizationcited by 16
  11. 2021Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networkscited by 14
  12. 2018Principled Deep Neural Network Training through Linear Programmingcited by 13
8 more
  1. 2024Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Timecited by 11
  2. 2023Optimal Sets and Solution Paths of ReLU Networkscited by 10
  3. 2021Parallel Deep Neural Networks Have Zero Duality Gapcited by 9
  4. 2023The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Modelscited by 7
  5. 2021Practical Convex Formulation of Robust One-hidden-layer Neural Network Trainingcited by 4
  6. 2024Convex Formulations for Training Two-Layer ReLU Neural Networkscited by 4
  7. 2018Proximal Gradient Algorithms: Applications in Signal Processingcited by 3
  8. 2024A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Featurescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Online Learning on Hidden-Convex Losses via Algorithmic Equivalence: Optimal Regret, Geometric Barrier, and Bandit Feedbackcited by 0