When does training on downscaled images yield the same gradients?
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
Diffusion transformers deliver strong image generation, but their training cost grows superlinearly with resolution. Recent work justifies training or sampling at reduced resolution on a spectral premise: at high noise, a downscaled latent preserves almost the full surviving signal. Whether a downscaled step also preserves the native training gradient signal, however, has remained unresolved. We reduce how that signal changes under downscaling to two terms: a noise-dependent term governed by the downscale ratio, which decays at high noise as the spectral premise predicts, and a σ-independent floor governed by the target grid's absolute token count, carried by the compute graph itself and removed by no noise level. The measured (route, σ) map corroborates the account and uncovers structure the spectral picture cannot express: on the 1024->768 route, a window (0.65 < σ < 0.95), predicted by no spectral criterion at any tolerance, where the downscaled gradient stays within a small margin of the native one. Training LoRA adapters with downscaled steps restricted to the routes and noise windows the map validates reduces training time by 14.6% at a fixed step budget while remaining near-native in weight space. Code is available at https://github.com/sorryhyun/animalora.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 20 of its 22 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
20 of 22 references.
Closest ancestors · 9
Same direction, newest first
- 2026Spectral Progressive Diffusion for Efficient Image and Video Generation18 May
- 2026Spectrally-Guided Diffusion Noise Schedules19 Mar
- 2026TIDE: Text-Informed Dynamic Extrapolation with Step-Aware Temperature Control for Diffusion Transformers9 Mar
- 2025DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion23 Oct
- 2025Scale-wise Distillation of Diffusion Models20 Mar
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
- 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
- 2022Scalable Diffusion Models with Transformers19 Dec
Other references · 11
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2025Qwen-Image Technical Reportcited by 973
- 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
- 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
- 2025Information Entropy Invariance: Enhancing Length Extrapolation in Attention Mechanismscited by 4
- 2026Uncertainty in a Single Pass: A Closed-Form Identity for One-Step Flow Matchingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Training-free Diffusion Acceleration with Bottleneck Sampling
- 2026Residual-Stream Burden Shapes Representation Learning in Diffusion Transformers
- 2026Energy-Conditioned Noise Schedule and Whitening for Spectral Diffusion
- 2026LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling
- 2026Correcting Guided Diffusion Trajectories with Spectral Alignment
- 2026Scale Space Diffusion
- 2025An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models
- 2026Revisiting Spectral Representations in Generative Diffusion Models