mlsanityData to 14 Oct 2026

Practical Convex Formulation of Robust One-hidden-layer Neural Network Training

arXiv 2105.12237 ↗First posted 25 May 2021cs.LG cs.CC stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that the training of a one-hidden-layer, scalar-output fully-connected ReLU neural network can be reformulated as a finite-dimensional convex program. Unfortunately, the scale of such a convex program grows exponentially in data size. In this work, we prove that a stochastic procedure with a linear complexity well approximates the exact formulation. Moreover, we derive a convex optimization approach to efficiently solve the "adversarial training" problem, which trains neural networks that are robust to adversarial input perturbations. Our method can be applied to binary classification and regression, and provides an alternative to the current adversarial training methods, such as Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD). We demonstrate in experiments that the proposed method achieves a noticeably better adversarial robustness and performance than the existing methods.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 51 references that are in the map. First posted on arXiv 25 May 2021.

Builds on

27 of 51 references.

References · 27

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  7. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  9. 2016End to End Learning for Self-Driving Carscited by 1,009
  10. 2016Adversarial Machine Learning at Scalecited by 936
  11. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  12. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
15 more
  1. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  2. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  3. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  4. 2017Adversarial Attacks on Neural Network Policiescited by 320
  5. 2018Certified Defenses against Adversarial Examplescited by 306
  6. 2017A Rewriting System for Convex Optimization Problemscited by 179
  7. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 98
  8. 2015Learning with a Strong Adversarycited by 68
  9. 2017Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputscited by 61
  10. 2020Random Smoothing Might be Unable to Certify ℓ_∞ Robustness for High-Dimensional Imagescited by 35
  11. 2020Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithmscited by 34
  12. 2020The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutionscited by 30
  13. 2020Tightened Convex Relaxations for Neural Network Robustness Certificationcited by 14
  14. 2020Data-Driven Certification of Neural Networks with Random Input Noisecited by 8
  15. 2020A Sequential Framework Towards an Exact SDP Verification of Neural Networkscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2022Efficient Global Optimization of Two-Layer ReLU Networks: Quadratic-Time Algorithms and Adversarial Trainingcited by 15
  2. 2023Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Offcited by 2
  3. 2024Adversarial Training of Two-Layer Polynomial and ReLU Activation Networks via Convex Optimizationcited by 1
  4. 2026Unveiling Hidden Convexity in Deep Learning: a Sparse Signal Processing Perspectivecited by 1