mlsanityData to 14 Oct 2026

Exact Convex Reformulations of Linear Neural Networks via Completely Positive Lifting

arXiv 2605.17692 ↗First posted 17 May 2026cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

We show that the training problem of a deep linear neural network under the squared loss admits an exact convex reformulation in a lifted space over a generalized completely positive cone. The reformulation has the same optimal value as the original nonconvex problem and is linear in the lifted variables, with all nonconvexity encoded in the cone constraint. Its ambient lifted dimension depends only on the input and output dimensions, independent of the network depth and the number of data points, and the bottleneck width enters only through scalar constraints. The construction proceeds by reducing the multilayer parameterization to a bilinear factorization, lifting it to a rank-constrained semidefinite program, expressing the rank constraint via a complementarity condition, and applying a completely positive lifting. The resulting formulation gives a conic representation of the nonconvexity induced by linear factorization and connects linear neural network training with copositive programming.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

18 of 31 references.

References · 18

  1. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  2. 2019Implicit Regularization in Deep Matrix Factorizationcited by 405
  3. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  4. 2016Deep Learning without Poor Local Minimacited by 234
  5. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 98
  6. 2020Revealing the Structure of Deep Neural Networks via Convex Dualitycited by 54
  7. 2020Convex Geometry and Duality of Over-parameterized Neural Networkscited by 49
  8. 2022Rank Diminishing in Deep Neural Networkscited by 47
  9. 2020Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithmscited by 34
  10. 2022Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformerscited by 30
  11. 2021Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programscited by 25
  12. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 18
6 more
  1. 2019Convex Formulation of Overparameterized Deep Neural Networkscited by 16
  2. 2022Duality for Neural Networks through Reproducing Kernel Banach Spacescited by 15
  3. 2023The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Modelscited by 7
  4. 2021Training Quantized Neural Networks to Global Optimality via Semidefinite Programmingcited by 6
  5. 2024Convex Formulations for Training Two-Layer ReLU Neural Networkscited by 4
  6. 2022Approximation hierarchies for copositive cone over symmetric cone and their comparisoncited by 1

Built on by

Not cited yet.