mlsanityData to 7 Oct 2026

Inf-DiT: Upsampling Any-Resolution Image with Memory-Efficient Diffusion Transformer

arXiv 2405.04312 ↗First posted 7 May 2024cs.CVcited by 14

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Diffusion models have shown remarkable performance in image generation in recent years. However, due to a quadratic increase in memory during generating ultra-high-resolution images (e.g. 4096*4096), the resolution of generated images is often limited to 1024*1024. In this work. we propose a unidirectional block attention mechanism that can adaptively adjust the memory overhead during the inference process and handle global dependencies. Building on this module, we adopt the DiT structure for upsampling and develop an infinite super-resolution model capable of upsampling images of various shapes and resolutions. Comprehensive experiments show that our model achieves SOTA performance in generating ultra-high-resolution images in both machine and human evaluation. Compared to commonly used UNet structures, our model can save more than 5x memory when generating 4096*4096 images. The project URL is https://github.com/THUDM/Inf-DiT.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 30 of its 33 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

30 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior29 Aug
  2. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization28 Aug
  3. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
15 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  5. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  6. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  7. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  8. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  9. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  10. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  11. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 524
  12. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Modelscited by 127
  13. 2023DemoFusion: Democratising High-Resolution Image Generation With No cited by 105
  14. 2022Any-resolution Training for High-resolution Image Synthesiscited by 72
  15. 2023Mixture of Diffusers for scene composition and high resolution image generationcited by 56

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  2. 2024Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Modelscited by 14
  3. 2025Audio Super-Resolution with Latent Bridge Modelscited by 13
  4. 2024Diffusion Models Need Visual Priors for Image Generationcited by 12
  5. 2025HiCache: A Plug-in Scaled-Hermite Upgrade for Taylor-Style Cache-then-Forecast Diffusion Accelerationcited by 8
  6. 2025InfGen: A Resolution-Agnostic Paradigm for Scalable Image Synthesiscited by 4
  7. 2025LR0.FM: Low-Res Benchmark and Improving Robustness for Zero-Shot Classification in Foundation Modelscited by 3
  8. 2025M3amba: CLIP-driven Mamba Model for Multi-modal Remote Sensing Classificationcited by 2
  9. 2025QDM: Quadtree-Based Region-Adaptive Sparse Diffusion Models for Efficient Image Super-Resolutioncited by 2
  10. 2026CASR: A Robust Cyclic Framework for Arbitrary Large-Scale Super-Resolution with Distribution Alignment and Self-Similarity Awarenesscited by 1
  11. 2025HaineiFRDM: Structure-Preserving Diffusion for Film Restoration under Fast Motion and Diverse Defectscited by 0
  12. 2026RPT-SR: Regional Prior attention Transformer for infrared image Super-Resolutioncited by 0
2 more
  1. 2026Optimizing ID Consistency in Multimodal Large Models: Facial Restoration via Alignment, Entanglement, and Disentanglementcited by 0
  2. 2026Tuning-Free Latent Diffusion Models for Ultrahigh-Resolution Image Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution
  2. 2024Effective Diffusion Transformer Architecture for Image Super-Resolution
  3. 2026HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models
  4. 2026DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression
  5. 2024ToDo: Token Downsampling for Efficient Generation of High-Resolution Images
  6. 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Step
  7. 2024FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion
  8. 2026TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution