mlsanityData to 7 Oct 2026

On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization

arXiv 2406.04558 ↗First posted 7 Jun 2024cs.LG math.OCcited by 12

Not on the map. Why?

Abstract (arXiv)

Constrained optimization offers a powerful framework to prescribe desired behaviors in neural network models. Typically, constrained problems are solved via their min-max Lagrangian formulations, which exhibit unstable oscillatory dynamics when optimized using gradient descent-ascent. The adoption of constrained optimization techniques in the machine learning community is currently limited by the lack of reliable, general-purpose update schemes for the Lagrange multipliers. This paper proposes the nuPI algorithm and contributes an optimization perspective on Lagrange multiplier updates based on PI controllers, extending the work of Stooke, Achiam and Abbeel (2020). We provide theoretical and empirical insights explaining the inability of momentum methods to address the shortcomings of gradient descent-ascent, and contrast this with the empirical success of our proposed nuPI controller. Moreover, we prove that nuPI generalizes popular momentum methods for single-objective minimization. Our experiments demonstrate that nuPI reliably stabilizes the multiplier dynamics and its hyperparameters enjoy robust and predictable behavior.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 26 of its 50 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

26 of 50 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2021Convex optimizationcited by 2,479
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  6. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  7. 2019The State of Sparsity in Deep Neural Networkscited by 471
  8. 2017Algorithmic decision making and the cost of fairnesscited by 402
  9. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  10. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 266
  11. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
  12. 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 142
14 more
  1. 2014Analysis and Design of Optimization Algorithms via Integral Quadratic Constraintscited by 105
  2. 2018Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goalscited by 103
  3. 2020Lagrangian Duality for Constrained Deep Learningcited by 72
  4. 2018Negative Momentum for Improved Game Dynamicscited by 59
  5. 2023Benchmarking Neural Network Training Algorithmscited by 46
  6. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  7. 2022Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraintscited by 25
  8. 2022A Lagrangian Duality Approach to Active Learningcited by 24
  9. 2018A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Accelerationcited by 15
  10. 2020On the Suboptimality of Negative Momentum for Minimax Optimizationcited by 11
  11. 2017Control Interpretations for First-Order Optimization Methodscited by 8
  12. 2023Balancing Act: Constraining Disparate Impact in Sparse Modelscited by 7
  13. 2022Neural Networks with Quantization Constraintscited by 4
  14. 2023A Control Theoretical Approach to Online Constrained Optimizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 3.

12 papers.

  1. 2025Cooper: A Library for Constrained Optimization in Deep Learningcited by 10
  2. 2024Embedding Safety into RL: A New Take on Trust Region Methodscited by 9
  3. 2025Composition and Alignment of Diffusion Models using Constrained Learningcited by 8
  4. 2025Feasible Learningcited by 5
  5. 2025Position: Adopt Constraints Over Fixed Penalties in Deep Learningcited by 4
  6. 2025Central Path Proximal Policy Optimizationcited by 3
  7. 2025Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguisecited by 1
  8. 2025Hyperflux: Pruning Reveals Importancecited by 0
  9. 2025Certifiable Safe RLHF: Semantic Grounding and Fixed Penalty Constraint Optimization for Safer LLM Alignmentcited by 0
  10. 2026Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Makingcited by 0
  11. 2026Neural Slack Variables for Shape Constraintscited by 0
  12. 2026Accelerated and Stable Convergence with Anchored Generalized Optimistic Methodcited by 0