# Memory-Computation Tradeoffs in Semi Amortized Parametric Optimization

arXiv: https://arxiv.org/abs/2607.20769 · HTML: https://mlsanity.com/paper/2607.20769/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 30 of 31)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Competitive caching with machine learned advice](https://mlsanity.com/paper/1802.05399/) (2018)
- [Algorithms with Predictions](https://mlsanity.com/paper/2006.09123/) (2020)
- [Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes](https://mlsanity.com/paper/1906.07697/) (2019)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [Semi-Amortized Variational Autoencoders](https://mlsanity.com/paper/1802.02550/) (2018)
- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Improving Online Algorithms via ML Predictions](https://mlsanity.com/paper/2407.17712/) (2024)
- [Learning an Approximate Model Predictive Controller with Guarantees](https://mlsanity.com/paper/1806.04167/) (2018)
- [From error bounds to the complexity of first-order descent methods for convex functions](https://mlsanity.com/paper/1510.08234/) (2015)
- [Learning to Solve Large-Scale Security-Constrained Unit Commitment Problems](https://mlsanity.com/paper/1902.01697/) (2019)
- [Online Mixed-Integer Optimization in Milliseconds](https://mlsanity.com/paper/1907.02206/) (2019)
- [Calculus of the exponent of Kurdyka-Lojasiewicz inequality and its applications to linear convergence of first-order methods](https://mlsanity.com/paper/1602.02915/) (2016)
- [Learning to Warm-Start Fixed-Point Optimization Algorithms](https://mlsanity.com/paper/2309.07835/) (2023)
- [Splitting methods with variable metric for KL functions](https://mlsanity.com/paper/1405.1357/) (2014)
- [End-to-End Learning to Warm-Start for Real-Time Quadratic Optimization](https://mlsanity.com/paper/2212.08260/) (2022)
- [Learning Predictions for Algorithms with Predictions](https://mlsanity.com/paper/2202.09312/) (2022)
- [Warm Start of Mixed-Integer Programs for Model Predictive Control of Hybrid Systems](https://mlsanity.com/paper/1910.08251/) (2019)
- [Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks](https://mlsanity.com/paper/1906.08257/) (2019)
- [Automatic nonlinear MPC approximation with closed-loop guarantees](https://mlsanity.com/paper/2312.10199/) (2023)
- [Neural Fixed-Point Acceleration for Convex Optimization](https://mlsanity.com/paper/2107.10254/) (2021)
- [Use of Machine Learning Models to Warmstart Column Generation for Unit Commitment](https://mlsanity.com/paper/2110.06872/) (2021)
- [Embedded Online Optimization for Model Predictive Control at Megahertz Rates](https://mlsanity.com/paper/1303.1090/) (2013)
- [Warm-starting active-set solvers using graph neural networks](https://mlsanity.com/paper/2511.13174/) (2025)

## Abstract (arXiv)

Learning-enabled decision systems often use offline data or computation to reduce online compute cost. Despite the empirical success of such approaches, there is limited general understanding of how much offline information is needed to achieve a desired accuracy under a fixed online computation budget. We study this question through the lens of amortized parametric optimization: an offline phase stores a finite memory of solved problem instances, and an online phase produces a solution to a new instance by retrieving a warm start and applying K steps of projected gradient descent. We analyze this setup for smooth convex parametric optimization over a compact domain, using a nonparametric predictor built from the stored offline solutions. For μ-strongly convex objectives, we establish matching upper and lower bounds on the memory required to guarantee varepsilon-accuracy under a fixed online iteration budget K. For convex objectives satisfying a β-growth condition (β>2), we obtain near-matching bounds and identify a phase transition in K beyond which additional memory provides no benefit. We further provide a general proof framework that (i) explicitly quantifies the memory cost of acceleration---how much offline memory is required to achieve a prescribed speedup over the unaided online optimizer---and (ii) identifies two key quantities driving this cost: the convergence rate of the online optimizer and the Lipschitz sensitivity of the solution map to the problem parameter. Experiments on parameterized ridge regression confirm the predicted memory--computation--accuracy tradeoffs.