mlsanityData to 7 Oct 2026

NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning

arXiv 2210.00973 ↗First posted 3 Oct 2022cs.LG cs.CV cs.MScited by 7

Not on the map. Why?

Abstract (arXiv)

Imposing explicit constraints is relatively new but increasingly pressing in deep learning, stimulated by, e.g., trustworthy AI that performs robust optimization over complicated perturbation sets and scientific applications that need to respect physical laws and constraints. However, it can be hard to reliably solve constrained deep learning problems without optimization expertise. The existing deep learning frameworks do not admit constraints. General-purpose optimization packages can handle constraints but do not perform auto-differentiation and have trouble dealing with nonsmoothness. In this paper, we introduce a new software package called NCVX, whose initial release contains the solver PyGRANSO, a PyTorch-enabled general-purpose optimization package for constrained machine/deep learning problems, the first of its kind. NCVX inherits auto-differentiation, GPU acceleration, and tensor variables from PyTorch, and is built on freely available and widely used open-source frameworks. NCVX is available at https://ncvx.org, with detailed documentation and numerous examples from machine/deep learning and other fields.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 56 of its 81 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

56 of 81 references.

References · 56

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  11. 2017Deep Image Priorcited by 1,097
  12. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
38 more
  1. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  2. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  3. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  4. 2019On Evaluating Adversarial Robustnesscited by 442
  5. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  6. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  7. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  8. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 286
  9. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  10. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 231
  11. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  12. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
  13. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 156
  14. 2019Branch and Bound for Piecewise Linear Neural Network Verificationcited by 138
  15. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  16. 2019Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Groupcited by 133
  17. 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 112
  18. 2018Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defensescited by 109
  19. 2020Geoopt: Riemannian Optimization in PyTorchcited by 109
  20. 2016Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiationcited by 93
  21. 2019Wasserstein Adversarial Examples via Projected Sinkhorn Iterationscited by 89
  22. 2018Semantic Adversarial Examplescited by 86
  23. 2017Orthogonal Recurrent Neural Networks with Scaled Cayley Transformcited by 82
  24. 2019Functional Adversarial Attackscited by 82
  25. 2018geomstats: a Python Package for Riemannian Geometry in Machine Learningcited by 71
  26. 2019Optimization for deep learning: theory and algorithmscited by 70
  27. 2015MLlib: Machine Learning in Apache Sparkcited by 55
  28. 2021Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraintscited by 50
  29. 2019Neural reparameterization improves structural optimizationcited by 44
  30. 2021Early Stopping for Deep Image Priorcited by 44
  31. 2019Fastened CROWN: Tightened Neural Network Robustness Certificatescited by 39
  32. 2016Recurrent Orthogonal Networks and Long-Memory Taskscited by 38
  33. 2020Training neural networks under physical constraints using a stochastic augmented Lagrangian approachcited by 19
  34. 2016Generalized BackPropagation, 'Etude De Cas: Orthogonalitycited by 17
  35. 2018Subgradient Descent Learns Orthogonal Dictionariescited by 17
  36. 2018McTorch, a manifold optimization library for deep learningcited by 16
  37. 2021Self-Validation: Early Stopping for Single-Instance Deep Generative Priorscited by 15
  38. 2019GENO -- GENeric Optimization for Classical Machine Learningcited by 12

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 1.

7 papers.

  1. 2023Equal Opportunity of Coverage in Fair Regressioncited by 16
  2. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  3. 2023Optimization and Optimizers for Adversarial Robustnesscited by 6
  4. 2025Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classificationcited by 2
  5. 2024Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processescited by 1
  6. 2024On the Laplace Approximation as Model Selection Criterion for Gaussian Processescited by 0
  7. 2026REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinationscited by 0