mlsanityData to 7 Oct 2026

Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems

arXiv 2211.15943 ↗First posted 29 Nov 2022math.OC stat.CO stat.MLcited by 21

Not on the map. Why?

Abstract (arXiv)

We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints. We consider a fully stochastic setting, where at each step a single sample is generated to estimate the objective gradient. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices (i.e., Hessians without modification) in SQP subproblems. As a trust-region method for constrained optimization, our algorithm must address an infeasibility issue -- the linearized equality constraints and trust-region constraints may lead to infeasible SQP subproblems. In this regard, we propose an adaptive relaxation technique to compute the trial step, consisting of a normal step and a tangential step. To control the lengths of these two steps while ensuring a scale-invariant property, we adaptively decompose the trust-region radius into two segments, based on the proportions of the rescaled feasibility and optimality residuals to the rescaled full KKT residual. The normal step has a closed form, while the tangential step is obtained by solving a trust-region subproblem, to which a solution ensuring the Cauchy reduction is sufficient for our study. We establish a global almost sure convergence guarantee for TR-StoSQP, and illustrate its empirical performance on both a subset of problems in the CUTEst test set and constrained logistic regression problems using data from the LIBSVM collection.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 17 of its 43 references that are in the map. First posted on arXiv 29 Nov 2022.

Builds on

17 of 43 references.

References · 17

  1. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  2. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  3. 2019Unified Optimal Analysis of the (Stochastic) Gradient Methodcited by 81
  4. 2020Sequential Quadratic Optimization for Nonlinear Equality Constrained Stochastic Optimizationcited by 30
  5. 2015Stochastic Optimization Using a Trust-Region Method and Random Modelscited by 25
  6. 2021An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangianscited by 25
  7. 2021A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobianscited by 21
  8. 2021Worst-Case Complexity of an SQP Method for Nonlinear Equality Constrained Stochastic Optimizationcited by 19
  9. 2021Inequality Constrained Stochastic Nonlinear Optimization via Active-Set Sequential Quadratic Programmingcited by 16
  10. 2022Hessian Averaging in Stochastic Newton Methods Achieves Superlinear Convergencecited by 14
  11. 2022Accelerating Stochastic Sequential Quadratic Programming for Equality Constrained Optimization using Predictive Variance Reductioncited by 13
  12. 2021Inexact Sequential Quadratic Optimization for Minimizing a Stochastic Objective Function Subject to Deterministic Nonlinear Equality Constraintscited by 11
5 more
  1. 2022An Adaptive Sampling Sequential Quadratic Programming Method for Equality Constrained Stochastic Optimizationcited by 11
  2. 2017A Stochastic Trust Region Algorithm Based on Careful Step Normalizationcited by 10
  3. 2019A Fully Stochastic Second-Order Trust Region Methodcited by 8
  4. 2017Exploiting Negative Curvature in Deterministic and Stochastic Optimizationcited by 7
  5. 2020Convergence Analysis of Accelerated Stochastic Gradient Descent under the Growth Conditioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 7; 2026: 5.

21 papers.

Sort:

By area:

  1. 2024Scaling physics-informed hard constraints with mixture-of-expertscited by 24
  2. 2024Variance-reduced first-order methods for deterministically constrained stochastic nonconvex optimization with strong convergence guaranteescited by 11
  3. 2023A Stochastic-Gradient-based Interior-Point Algorithm for Solving Smooth Bound-Constrained Optimization Problemscited by 8
  4. 2025End-to-End Probabilistic Framework for Learning with Hard Constraintscited by 8
  5. 2023Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketchingcited by 5
  6. 2023Almost-sure convergence of iterates and multipliers in stochastic sequential quadratic optimizationcited by 5
  7. 2024Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Modelscited by 5
  8. 2025Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraintscited by 5
  9. 2025High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noisecited by 4
  10. 2026A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad and Muoncited by 2
  11. 2024Single-Loop Deterministic and Stochastic Interior-Point Algorithms for Nonlinearly Constrained Optimizationcited by 1
  12. 2024A Trust-Region Algorithm for Noisy Equality Constrained Optimizationcited by 1
9 more
  1. 2024Stochastic interior-point methods for smooth conic optimization with applicationscited by 1
  2. 2026A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Methodcited by 1
  3. 2025Mirror Descent Linearized Augmented Lagrangian Methods for Nonconvex Constrained Stochastic Zeroth-Order Optimizationcited by 0
  4. 2025Online Statistical Inference of Constrained Stochastic Optimization via Random Scalingcited by 0
  5. 2025Benchmarking Stochastic Approximation Algorithms for Fairness-Constrained Training of Deep Neural Networkscited by 0
  6. 2025humancompatible.train: Implementing Optimization Algorithms for Stochastically-Constrained Stochastic Optimization Problemscited by 0
  7. 2026TRSVR: An Adaptive Stochastic Trust-Region Method with Variance Reductioncited by 0
  8. 2026Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Makingcited by 0
  9. 2026TR-SSQP: A Trust-Region Method for Constrained Stochastic Optimization under Heavy-Tailed Noisecited by 0