mlsanityData to 7 Oct 2026

Reviving ConvNeXt for Efficient Convolutional Diffusion Models

arXiv 2603.09408 ↗First posted 10 Mar 2026cs.CV cs.AI cs.LGcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recent diffusion models increasingly favor Transformer backbones, motivated by the remarkable scalability of fully attentional architectures. Yet the locality bias, parameter efficiency, and hardware friendliness--the attributes that established ConvNets as the efficient vision backbone--have seen limited exploration in modern generative modeling. Here we introduce the fully convolutional diffusion model (FCDM), a model having a backbone similar to ConvNeXt, but designed for conditional diffusion modeling. We find that using only 50% of the FLOPs of DiT-XL/2, FCDM-XL achieves competitive performance with 7× and 7.5× fewer training steps at 256×256 and 512×512 resolutions, respectively. Remarkably, FCDM-XL can be trained on a 4-GPU system, highlighting the exceptional training efficiency of our architecture. Our results demonstrate that modern convolutional designs provide a competitive and highly efficient alternative for scaling diffusion models, reviving ConvNeXt as a simple yet powerful building block for efficient generative modeling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 63 of its 74 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

63 of 74 references.

Closest ancestors · 28

Same direction, newest first

  1. 2025DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling16 May
  2. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models2 Jan
  3. 2024DiC: Rethinking Conv3x3 Designs in Diffusion Models31 Dec
  4. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion25 Oct
  5. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification14 Oct
  6. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think9 Oct
  7. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers4 May
  8. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformer25 Mar
  9. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation7 Mar
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  11. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  12. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
16 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  4. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  5. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  6. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  7. 2022Scalable Diffusion Models with Transformers19 Dec
  8. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  9. 2022Classifier-Free Diffusion Guidance26 Jul
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  13. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  14. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  15. 2020Denoising Diffusion Implicit Models6 Oct
  16. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2016Densely Connected Convolutional Networkscited by 6,691
23 more
  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Layer Normalizationcited by 5,579
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2022Flow Matching for Generative Modelingcited by 5,426
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  13. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  14. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  15. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  16. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  17. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  18. 2021Generating Images with Sparse Representationscited by 262
  19. 2022Neighborhood Attention Transformercited by 235
  20. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  21. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 85
  22. 2024DiG: Scalable and Efficient Diffusion Models with Gated Linear Attentioncited by 56
  23. 2024SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Trainingcited by 27

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Exploring Vision Transformers as Diffusion Learners
  2. 2024SparseDM: Toward Sparse Efficient Diffusion Models
  3. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning
  4. 2024Diffscaler: Enhancing the Generative Prowess of Diffusion Transformers
  5. 2023Neural Diffusion Models
  6. 2024Diffusion Models in Low-Level Vision: A Survey
  7. 2024Scalable Diffusion Models with State Space Backbone
  8. 2024Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models