# Multi-ResNets for Subspace Preconditioning in Constrained Optimization

arXiv: https://arxiv.org/abs/2606.06300 · HTML: https://mlsanity.com/paper/2606.06300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Gaussian Process Behaviour in Wide Deep Neural Networks](https://mlsanity.com/paper/1804.11271/) (2018)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [The Power Grid Library for Benchmarking AC Optimal Power Flow Algorithms](https://mlsanity.com/paper/1908.02788/) (2019)
- [A Unified Framework for U-Net Design and Analysis](https://mlsanity.com/paper/2305.19638/) (2023)
- [FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees](https://mlsanity.com/paper/2506.00362/) (2025)
- [PGLearn -- An Open-Source Learning Toolkit for Optimal Power Flow](https://mlsanity.com/paper/2505.22825/) (2025)
- [A Multi-Resolution Framework for U-Nets with Applications to Hierarchical VAEs](https://mlsanity.com/paper/2301.08187/) (2023)
- [QCQP-Net: Reliably Learning Feasible Alternating Current Optimal Power Flow Solutions Under Constraints](https://mlsanity.com/paper/2401.06820/) (2024)

## Abstract (arXiv)

We propose MResOpt, a staged residual neural network architecture for constrained optimization problems. Our architecture fits within predict-complete-correct pipelines and decomposes constraint satisfaction by priority through intermediate re-completion and stage-aware losses. The framework enables domain-informed ordered constraint satisfaction which allows the network to utilize ordinal structure when present. Under an idealized infinite-width regime, we show that our design behaves as sequential Gaussian Process regression. On synthetic QP, QCQP, and SOCP benchmarks, the staged architecture improves high-priority constraint satisfaction across convex and non-convex settings. On line-flow-constrained AC optimal power flow, we introduce a physics-motivated constraint ordering and show that MResOpt supports a learned division of labor that keeps iterates on the equality manifold, achieving substantially lower high-priority violation than reprojected baselines while remaining computationally efficient.