mlsanityData to 7 Oct 2026

Tutorial on amortized optimization

arXiv 2202.00665 ↗First posted 1 Feb 2022cs.LG cs.AI math.OCcited by 83

Not on the map. Why?

Abstract (arXiv)

Optimization is a ubiquitous modeling tool and is often deployed in settings which repeatedly solve similar instances of the same problem. Amortized optimization methods use learning to predict the solutions to problems in these settings, exploiting the shared structure between similar problem instances. These methods have been crucial in variational inference and reinforcement learning and are capable of solving optimization problems many orders of magnitudes faster than traditional optimization methods that do not use amortization. This tutorial presents an introduction to the amortized optimization foundations behind these advancements and overviews their applications in variational inference, sparse coding, gradient-based meta-learning, control, reinforcement learning, convex optimization, optimal transport, and deep equilibrium networks. The source code for this tutorial is available at https://github.com/facebookresearch/amortized-optimization-tutorial.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). Builds on 190 of its 348 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

190 of 348 references.

References · 190

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2016Layer Normalizationcited by 5,579
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  11. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  12. 2013Auto-Encoding Variational Bayescited by 3,980
38 more
  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  3. 2021Convex optimizationcited by 2,479
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  6. 2015Variational Inference with Normalizing Flowscited by 2,161
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  9. 2016Variational Inference: A Review for Statisticianscited by 1,466
  10. 2017Deep Setscited by 1,394
  11. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  12. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  13. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  14. 2019An Introduction to Variational Autoencoderscited by 997
  15. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  16. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  17. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  18. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  19. 2016Learning to learn by gradient descent by gradient descentcited by 616
  20. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  21. 2019Deep Equilibrium Modelscited by 603
  22. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  23. 2014Convex Optimization: Algorithms and Complexitycited by 569
  24. 2017Learning Combinatorial Optimization Algorithms over Graphscited by 557
  25. 2019Meta-Learning with Implicit Gradientscited by 523
  26. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 516
  27. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  28. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  29. 2019Differentiable Convex Optimization Layerscited by 458
  30. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  31. 2018Conditional Neural Processescited by 421
  32. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  33. 2016Input Convex Neural Networkscited by 407
  34. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 407
  35. 2016Tutorial on Variational Autoencoderscited by 404
  36. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  37. 2018Meta-learning with differentiable closed-form solverscited by 395
  38. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362

50 most cited of 190 All 190 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 25; 2025: 21; 2026: 30.

83 papers.

Sort:

By area:

  1. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  2. 2026Learning Latent Action World Models In The Wildcited by 42
  3. 2024Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifoldcited by 41
  4. 2024Online Adaptation of Language Models with a Memory of Amortized Contextscited by 35
  5. 2023Learning to Warm-Start Fixed-Point Optimization Algorithmscited by 32
  6. 2024Dynamic Conditional Optimal Transport through Simulation-Free Flowscited by 24
  7. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  8. 2024DiffuSolve: Diffusion-based Solver for Non-convex Trajectory Optimizationcited by 19
  9. 2025FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guaranteescited by 19
  10. 2023Neural incomplete factorization: learning preconditioners for the conjugate gradient methodcited by 18
  11. 2024AToM: Amortized Text-to-Mesh using 2D Diffusioncited by 18
  12. 2025Pinet: Optimizing hard-constrained neural networks with orthogonal projection layerscited by 17
38 more
  1. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  2. 2024Neural Optimal Transport with Lagrangian Costscited by 16
  3. 2025A Survey on Active Feature Acquisition Strategiescited by 16
  4. 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
  5. 2024Poisson Variational Autoencodercited by 11
  6. 2023Generalization Bound and Learning Methods for Data-Driven Projections in Linear Programmingcited by 10
  7. 2024Scaling Laws for the Value of Individual Data Points in Machine Learningcited by 10
  8. 2025PABBO: Preferential Amortized Black-Box Optimizationcited by 7
  9. 2025Diffusion LLMs are Natural Adversaries for any LLMcited by 7
  10. 2024Metric Learning to Accelerate Convergence of Operator Splitting Methods for Differentiable Parametric Programmingcited by 6
  11. 2025Aligning Diffusion Model with Problem Constraints for Trajectory Optimizationcited by 6
  12. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
  13. 2024Optimal Transport for Structure Learning Under Missing Datacited by 5
  14. 2024Data-Driven Performance Guarantees for Classical and Learned Optimizerscited by 5
  15. 2025Diffusion Policies for Generative Modeling of Spacecraft Trajectoriescited by 5
  16. 2025Provably data-driven projection method for quadratic programmingcited by 5
  17. 2025In-Context Multi-Objective Optimizationcited by 5
  18. 2024SPO: Sequential Monte Carlo Policy Optimisationcited by 4
  19. 2025Fast filtering of non-Gaussian models using Amortized Optimal Transport Mapscited by 4
  20. 2025LeMat-Traj: A Scalable and Unified Dataset of Materials Trajectories for Atomistic Modelingcited by 4
  21. 2025Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfoldingcited by 4
  22. 2023Differentiable Cutting-plane Layers for Mixed-integer Linear Optimizationcited by 3
  23. 2024On a Neural Implementation of Brenier's Polar Factorizationcited by 3
  24. 2024Self-Supervised Learning of Iterative Solvers for Constrained Optimizationcited by 3
  25. 2025Deep Learning Warm Starts for Trajectory Optimization on the International Space Stationcited by 3
  26. 2025Unsupervised Learning for Optimal Transport plan prediction between unbalanced graphscited by 3
  27. 2025Learning to accelerate distributed ADMM using graph neural networkscited by 3
  28. 2024Decision-Focused Forecasting: A Differentiable Multistage Optimisation Architecturecited by 2
  29. 2024From Learning to Optimize to Learning Optimization Algorithmscited by 2
  30. 2024Scalable unsupervised alignment of general metric and non-metric structurescited by 2
  31. 2024Reinforcement Learning applied to Insurance Portfolio Pursuitcited by 2
  32. 2025Sobolev Training of End-to-End Optimization Proxiescited by 2
  33. 2025Local Scale Equivariance with Latent Deep Equilibrium Canonicalizercited by 2
  34. 2025An Encoder-Decoder Network for Beamforming over Sparse Large-Scale MIMO Channelscited by 2
  35. 2025Warm-starting active-set solvers using graph neural networkscited by 2
  36. 2026Learning to accelerate Krasnosel'skii-Mann fixed-point iterations with guaranteescited by 2
  37. 2026Improving Feasibility via Fast Autoencoder-Based Projectionscited by 2
  38. 2026Amortized Optimal Transport from Sliced Potentialscited by 2

50 most cited of 83 All 83 (JSON)