mlsanityData to 7 Oct 2026

ToDo: Token Downsampling for Efficient Generation of High-Resolution Images

arXiv 2402.13573 ↗First posted 21 Feb 2024cs.CV cs.AI cs.LGcited by 14

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Attention mechanism has been crucial for image diffusion models, however, their quadratic computational complexity limits the sizes of images we can process within reasonable time and memory constraints. This paper investigates the importance of dense attention in generative image models, which often contain redundant features, making them suitable for sparser attention mechanisms. We propose a novel training-free method ToDo that relies on token downsampling of key and value tokens to accelerate Stable Diffusion inference by up to 2x for common sizes and up to 4.5x or more for high resolutions like 2048x2048. We demonstrate that our approach outperforms previous methods in balancing efficient throughput and fidelity.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 13 of its 17 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

13 of 17 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  2. 2023Token Merging for Fast Stable Diffusion30 Mar
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2020Denoising Diffusion Implicit Models6 Oct

Other references · 9

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  5. 2021Transformers in Vision: A Surveycited by 1,009
  6. 2022Token Merging: Your ViT But Fastercited by 995
  7. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  8. 2023A Survey on Efficient Training of Transformerscited by 37
  9. 2020AReLU: Attention-based Rectified Linear Unitcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 7.

14 papers.

Sort:

By area:

  1. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  2. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformerscited by 51
  3. 2024Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformerscited by 21
  4. 2024Importance-Based Token Merging for Efficient Image and Video Generationcited by 20
  5. 2025DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformerscited by 19
  6. 2025RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancycited by 7
  7. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deploymentcited by 4
  8. 2026DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformerscited by 3
  9. 2026CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformerscited by 2
  10. 2026BiGain: Unified Token Compression for Joint Generation and Classificationcited by 0
  11. 2026Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awarenesscited by 0
  12. 2026HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editingcited by 0
2 more
  1. 2026Tuning-Free Latent Diffusion Models for Ultrahigh-Resolution Image Editingcited by 0
  2. 2026Importance-Aware OBS Pruning for Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Accelerating Diffusion Decoders via Multi-Scale Sampling and One-Step Distillation
  2. 2025Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
  4. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  5. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models
  6. 2026Denoising Diffusion Generative Models Secretly Calculate Attentions
  7. 2024RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation
  8. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models