mlsanityData to 7 Oct 2026

Neural Network Diffusion

arXiv 2402.13144 ↗First posted 20 Feb 2024cs.LG cs.CVcited by 39

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also textitgenerate high-performing neural network parameters. Our approach is simple, utilizing an autoencoder and a diffusion model. The autoencoder extracts latent representations of a subset of the trained neural network parameters. Next, a diffusion model is trained to synthesize these latent representations from random noise. This model then generates new representations, which are passed through the autoencoder's decoder to produce new subsets of high-performing network parameters. Across various architectures and datasets, our approach consistently generates models with comparable or improved performance over trained networks, with minimal additional cost. Notably, we empirically find that the generated models are not memorizing the trained ones. Our results encourage more exploration into the versatile use of diffusion models. Our code is available hrefhttps://github.com/NUS-HPC-AI-Lab/Neural-Network-Diffusionhere.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 51 of its 81 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

51 of 81 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning31 Jul
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  4. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
31 more
  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2015Variational Inference with Normalizing Flowscited by 2,161
  13. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  14. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  15. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  16. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  17. 2016HyperNetworkscited by 1,003
  18. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  19. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  20. 2013Predicting Parameters in Deep Learningcited by 223
  21. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  22. 2020FCOS: A simple and strong anchor-free object detectorcited by 171
  23. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  24. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functionscited by 133
  25. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
  26. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  27. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  28. 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 85
  29. 2022Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weightscited by 63
  30. 2022HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learningcited by 55
  31. 2023Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?cited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 18; 2026: 11.

39 papers.

Sort:

By area:

  1. 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
  2. 2025Symmetry in Neural Network Parameter Spacescited by 31
  3. 2024A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Trainingcited by 30
  4. 2025Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weightscited by 18
  5. 2024Conditional LoRA Parameter Generationcited by 16
  6. 2024Diffusion Models are Evolutionary Algorithmscited by 15
  7. 2025Recurrent Diffusion for Large-Scale Parameter Generationcited by 15
  8. 2025In-Context Meta LoRA Generationcited by 14
  9. 2025Generative Modeling of Weights: Generalization or Memorization?cited by 12
  10. 2024Unsupervised Model Tree Heritage Recoverycited by 11
  11. 2024Text-to-Model: Text-Conditioned Neural Network Diffusion for Train-Once-for-All Personalizationcited by 10
  12. 2024ElastoGen: 4D Generative Elastodynamicscited by 10
27 more
  1. 2024PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scorescited by 7
  2. 2024Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Modelscited by 4
  3. 2024WARPD: World model Assisted Reactive Policy Diffusioncited by 3
  4. 2026DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weightscited by 3
  5. 2025Few-shot Implicit Function Generation via Equivariancecited by 2
  6. 2026PILD: Physics-Informed Learning via Diffusioncited by 2
  7. 2026HY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editingcited by 2
  8. 2026Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformerscited by 2
  9. 2025Make Optimization Once and for All with Fine-grained Guidancecited by 1
  10. 2025NeuroGen: Neural Network Parameter Generation via Large Language Modelscited by 1
  11. 2025DiffPINN: Generative diffusion-initialized physics-informed neural networks for accelerating seismic wavefield representationcited by 1
  12. 2025Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarioscited by 1
  13. 2025On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMscited by 1
  14. 2025Weight Space Representation Learning via Neural Field Adaptationcited by 1
  15. 2024Puppet-CNN: Continuous Parameter Dynamics for Input-Adaptive Convolutional Networkscited by 0
  16. 2025Model Diffusion for Certifiable Few-shot Transfer Learningcited by 0
  17. 2025Fast Kernel-Space Diffusion for Remote Sensing Pansharpeningcited by 0
  18. 2025Reimagining Parameter Space Exploration with Diffusion Modelscited by 0
  19. 2025Learning with springs and stickscited by 0
  20. 2025Delta Activations: A Representation for Finetuned Large Language Modelscited by 0
  21. 2026Breaking the Scale Barrier: One-Shot Knowledge Transfer via Frequency Transformcited by 0
  22. 2026Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weightscited by 0
  23. 2026Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetworkcited by 0
  24. 2026WeightCLIP: Aligning Datasets and Models for Weight Space Learningcited by 0
  25. 2026Architecture Generalization with MetaNCAcited by 0
  26. 2026Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generationcited by 0
  27. 2026CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Representation Learning with Diffusion Models
  2. 2023Neural Diffusion Models
  3. 2025On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning
  4. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Models
  5. 2025Generalization of Diffusion Models Arises with a Balanced Representation Space
  6. 2025Diffusion Transformers with Representation Autoencoders
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Models
  8. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models