# On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization

arXiv: https://arxiv.org/abs/2406.04558 · HTML: https://mlsanity.com/paper/2406.04558/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [The State of Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1902.09574/) (2019)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems](https://mlsanity.com/paper/1906.00331/) (2019)
- [A Variational Inequality Perspective on Generative Adversarial Networks](https://mlsanity.com/paper/1802.10551/) (2018)
- [A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach](https://mlsanity.com/paper/1901.08511/) (2019)
- [Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints](https://mlsanity.com/paper/1408.3595/) (2014)
- [Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals](https://mlsanity.com/paper/1809.04198/) (2018)
- [Lagrangian Duality for Constrained Deep Learning](https://mlsanity.com/paper/2001.09394/) (2020)
- [Negative Momentum for Improved Game Dynamics](https://mlsanity.com/paper/1807.04740/) (2018)
- [Benchmarking Neural Network Training Algorithms](https://mlsanity.com/paper/2306.07179/) (2023)
- [Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimization](https://mlsanity.com/paper/2102.09468/) (2021)
- [Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints](https://mlsanity.com/paper/2208.04425/) (2022)
- [A Lagrangian Duality Approach to Active Learning](https://mlsanity.com/paper/2202.04108/) (2022)
- [A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration](https://mlsanity.com/paper/1808.03408/) (2018)
- [On the Suboptimality of Negative Momentum for Minimax Optimization](https://mlsanity.com/paper/2008.07459/) (2020)
- [Control Interpretations for First-Order Optimization Methods](https://mlsanity.com/paper/1703.01670/) (2017)
- [Balancing Act: Constraining Disparate Impact in Sparse Models](https://mlsanity.com/paper/2310.20673/) (2023)
- [Neural Networks with Quantization Constraints](https://mlsanity.com/paper/2210.15623/) (2022)
- [A Control Theoretical Approach to Online Constrained Optimization](https://mlsanity.com/paper/2309.15498/) (2023)

## Built on by (most cited first, 12 of 12)

- [Cooper: A Library for Constrained Optimization in Deep Learning](https://mlsanity.com/paper/2504.01212/) (2025)
- [Embedding Safety into RL: A New Take on Trust Region Methods](https://mlsanity.com/paper/2411.02957/) (2024)
- [Composition and Alignment of Diffusion Models using Constrained Learning](https://mlsanity.com/paper/2508.19104/) (2025)
- [Feasible Learning](https://mlsanity.com/paper/2501.14912/) (2025)
- [Position: Adopt Constraints Over Fixed Penalties in Deep Learning](https://mlsanity.com/paper/2505.20628/) (2025)
- [Central Path Proximal Policy Optimization](https://mlsanity.com/paper/2506.00700/) (2025)
- [Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise](https://mlsanity.com/paper/2509.22500/) (2025)
- [Hyperflux: Pruning Reveals Importance](https://mlsanity.com/paper/2504.05349/) (2025)
- [Certifiable Safe RLHF: Semantic Grounding and Fixed Penalty Constraint Optimization for Safer LLM Alignment](https://mlsanity.com/paper/2510.03520/) (2025)
- [Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Making](https://mlsanity.com/paper/2606.07088/) (2026)
- [Neural Slack Variables for Shape Constraints](https://mlsanity.com/paper/2606.13803/) (2026)
- [Accelerated and Stable Convergence with Anchored Generalized Optimistic Method](https://mlsanity.com/paper/2606.21528/) (2026)

## Abstract (arXiv)

Constrained optimization offers a powerful framework to prescribe desired behaviors in neural network models. Typically, constrained problems are solved via their min-max Lagrangian formulations, which exhibit unstable oscillatory dynamics when optimized using gradient descent-ascent. The adoption of constrained optimization techniques in the machine learning community is currently limited by the lack of reliable, general-purpose update schemes for the Lagrange multipliers. This paper proposes the nuPI algorithm and contributes an optimization perspective on Lagrange multiplier updates based on PI controllers, extending the work of Stooke, Achiam and Abbeel (2020). We provide theoretical and empirical insights explaining the inability of momentum methods to address the shortcomings of gradient descent-ascent, and contrast this with the empirical success of our proposed nuPI controller. Moreover, we prove that nuPI generalizes popular momentum methods for single-objective minimization. Our experiments demonstrate that nuPI reliably stabilizes the multiplier dynamics and its hyperparameters enjoy robust and predictable behavior.