mlsanityData to 14 Oct 2026

Make Optimization Once and for All with Fine-grained Guidance

arXiv 2503.11462 ↗First posted 14 Mar 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks. L2O paradigms achieve great outcomes, e.g., refitting optimizer, generating unseen solutions iteratively or directly. However, conventional L2O methods require intricate design and rely on specific optimization processes, limiting scalability and generalization. Our analyses explore general framework for learning optimization, called Diff-L2O, focusing on augmenting sampled solutions from a wider view rather than local updates in real optimization process only. Meanwhile, we give the related generalization bound, showing that the sample diversity of Diff-L2O brings better performance. This bound can be simply applied to other fields, discussing diversity, mean-variance, and different tasks. Diff-L2O's strong compatibility is empirically verified with only minute-level training, comparing with other hour-levels.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 43 of its 53 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

43 of 53 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  11. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  12. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
31 more
  1. 2016Learning to learn by gradient descent by gradient descentcited by 620
  2. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  3. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 565
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  5. 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 266
  6. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  7. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  8. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  9. 2016Learning to Optimizecited by 195
  10. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  11. 2021Combinatorial Optimization with Physics-Inspired Graph Neural Networkscited by 108
  12. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  13. 2023Diffusion Models for Black-Box Optimizationcited by 76
  14. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  15. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  16. 2017Learning Gradient Descent: Better Generalization and Longer Horizonscited by 50
  17. 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 46
  18. 2022Denoising Likelihood Score Matching for Conditional Score-based Data Generationcited by 46
  19. 2024Neural Network Diffusioncited by 42
  20. 2020Automated Synthetic-to-Real Generalizationcited by 35
  21. 2019Learning to Optimize in Swarmscited by 23
  22. 2019First-order optimization algorithms via inertial systems with Hessian driven dampingcited by 22
  23. 2018Learning to Defend by Learning to Attackcited by 17
  24. 2020Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case Studycited by 16
  25. 2022Symbolic Learning to Optimize: Towards Interpretability and Scalabilitycited by 16
  26. 2025Recurrent Diffusion for Large-Scale Parameter Generationcited by 15
  27. 2022Transformer-Based Learned Optimizationcited by 13
  28. 2022Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networkscited by 9
  29. 2023Mnemosyne: Learning to Train Transformers with Transformerscited by 9
  30. 2019Twin Auxiliary Classifiers GANcited by 8
  31. 2024ODE-based Learning to Optimizecited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusioncited by 6