# Backpropagation of Unrolled Solvers with Folded Optimization

arXiv: https://arxiv.org/abs/2301.12047 · HTML: https://mlsanity.com/paper/2301.12047/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 28 Jan 2023.

## Builds on (most cited first, 21 of 21)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization](https://mlsanity.com/paper/1607.05447/) (2016)
- [MIPaaL: Mixed Integer Program as a Layer](https://mlsanity.com/paper/1907.05912/) (2019)
- [Interior Point Solving for LP-based prediction+optimisation](https://mlsanity.com/paper/2010.13943/) (2020)
- [Differentiating Through a Cone Program](https://mlsanity.com/paper/1904.09043/) (2019)
- [Gradient Estimation with Stochastic Softmax Tricks](https://mlsanity.com/paper/2006.08063/) (2020)
- [Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization](https://mlsanity.com/paper/2205.02640/) (2022)
- [Smooth Loss Functions for Deep Top-k Classification](https://mlsanity.com/paper/1802.07595/) (2018)
- [The Limited Multi-Label Projection Layer](https://mlsanity.com/paper/1906.08707/) (2019)
- [Solution Refinement at Regular Points of Conic Problems](https://mlsanity.com/paper/1811.02157/) (2018)
- [End-to-end Learning for Fair Ranking Systems](https://mlsanity.com/paper/2111.10723/) (2021)
- [Differentiable Model Selection for Ensemble Learning](https://mlsanity.com/paper/2211.00251/) (2022)

## Built on by (most cited first, 13 of 13)

- [Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities](https://mlsanity.com/paper/2307.13565/) (2023)
- [FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees](https://mlsanity.com/paper/2506.00362/) (2025)
- [Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimization](https://mlsanity.com/paper/2311.13087/) (2023)
- [Global-Decision-Focused Neural ODEs for Proactive Grid Resilience Management](https://mlsanity.com/paper/2502.18321/) (2025)
- [Learning Joint Models of Prediction and Optimization](https://mlsanity.com/paper/2409.04898/) (2024)
- [End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty](https://mlsanity.com/paper/2402.07772/) (2024)
- [Analyzing and Enhancing the Backward-Pass Convergence of Unrolled Optimization](https://mlsanity.com/paper/2312.17394/) (2023)
- [Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted Averages](https://mlsanity.com/paper/2402.05252/) (2024)
- [End-to-End Fairness Optimization with Fair Decision-Focused Learning](https://mlsanity.com/paper/2607.29441/) (2026)
- [Learning-Augmented Power System Operations: A Unified Optimization View](https://mlsanity.com/paper/2505.05203/) (2025)
- [Bundle Network: a Machine Learning-Based Bundle Method](https://mlsanity.com/paper/2509.24736/) (2025)
- [Neural Ising Machines via Unrolling and Zeroth-Order Training](https://mlsanity.com/paper/2602.00302/) (2026)
- [Learning to Attack: A Bandit Approach to Adversarial Context Poisoning](https://mlsanity.com/paper/2603.00567/) (2026)

## Abstract (arXiv)

The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an optimization problem, which typically lacks a closed form. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. While flexible and general, unrolling can encounter accuracy and efficiency issues in practice. These issues can be avoided by analytical differentiation of the optimization, but current frameworks impose rigid requirements on the optimization problem's form. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation. Additionally, it proposes a unifying view of unrolling and analytical differentiation through optimization mappings. Experiments over various model-based learning tasks demonstrate the advantages of the approach both computationally and in terms of enhanced expressiveness.