# HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models

arXiv: https://arxiv.org/abs/2311.17528 · HTML: https://mlsanity.com/paper/2311.17528/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 37 of its 41 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Perception Prioritized Training of Diffusion Models](https://mlsanity.com/paper/2204.00227/) (2022)
- [Token Merging for Fast Stable Diffusion](https://mlsanity.com/paper/2303.17604/) (2023)
- [ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models](https://mlsanity.com/paper/2310.07702/) (2023)
- [Diffusion Probabilistic Model Made Slim](https://mlsanity.com/paper/2211.17106/) (2022)
- [SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions](https://mlsanity.com/paper/2306.05178/) (2023)
- [DemoFusion: Democratising High-Resolution Image Generation With No ](https://mlsanity.com/paper/2311.16973/) (2023)
- [DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2304.06648/) (2023)
- [Relay Diffusion: Unifying diffusion process across resolutions for image synthesis](https://mlsanity.com/paper/2309.03350/) (2023)
- [Effective Real Image Editing with Accelerated Iterative Diffusion Inversion](https://mlsanity.com/paper/2309.04907/) (2023)
- [Any-resolution Training for High-resolution Image Synthesis](https://mlsanity.com/paper/2204.07156/) (2022)

## Built on by (most cited first, 29 of 29)

- [FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusion](https://mlsanity.com/paper/2412.09626/) (2024)
- [MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning](https://mlsanity.com/paper/2408.11001/) (2024)
- [ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models](https://mlsanity.com/paper/2403.02084/) (2024)
- [DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion](https://mlsanity.com/paper/2510.20766/) (2025)
- [FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion](https://mlsanity.com/paper/2411.18552/) (2024)
- [HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streaming](https://mlsanity.com/paper/2512.21338/) (2025)
- [RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation](https://mlsanity.com/paper/2410.06055/) (2024)
- [ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion](https://mlsanity.com/paper/2510.25818/) (2025)
- [FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation](https://mlsanity.com/paper/2511.14712/) (2025)
- [UltraFlux: Data-Model Co-Design for High-quality Native 4K Text-to-Image Generation across Diverse Aspect Ratios](https://mlsanity.com/paper/2511.18050/) (2025)
- [FlexCache: Flexible Approximate Cache System for Video Diffusion](https://mlsanity.com/paper/2501.04012/) (2024)
- [Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment](https://mlsanity.com/paper/2508.06160/) (2025)
- [AccDiffusion v2: Towards More Accurate Higher-Resolution Diffusion Extrapolation](https://mlsanity.com/paper/2412.02099/) (2024)
- [LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding](https://mlsanity.com/paper/2503.04344/) (2025)
- [Exploring Position Encoding in Diffusion U-Net for Training-free High-resolution Image Generation](https://mlsanity.com/paper/2503.09830/) (2025)
- [UltraImageGen: Efficient Ultra-High-Resolution Image Generation with Hierarchical Local Attention](https://mlsanity.com/paper/2510.16325/) (2025)
- [GuessBench: Sensemaking Multimodal Creativity in the Wild](https://mlsanity.com/paper/2506.00814/) (2025)
- [CineScale: Tuning-Free High-Resolution Video Generation](https://mlsanity.com/paper/2508.15774/) (2025)
- [One Small Step in Latent, One Giant Leap for Pixels: Fast Latent Upscale Adapter for Your Diffusion Models](https://mlsanity.com/paper/2511.10629/) (2025)
- [Training-Free Inference for High-Resolution Sinogram Completion](https://mlsanity.com/paper/2506.08809/) (2025)
- [HiMat: DiT-based Ultra-High Resolution SVBRDF Generation](https://mlsanity.com/paper/2508.07011/) (2025)
- [Supersampling Stable Diffusion and Beyond: A Seamless, Training-Free Approach for Scaling Neural Networks Using Common Interpolation Methods](https://mlsanity.com/paper/2605.08698/) (2026)
- [ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models](https://mlsanity.com/paper/2605.10045/) (2026)
- [AtlasVid: Efficient Ultra-High-Resolution Long Video Generation via Decoupled Global-Local Modeling](https://mlsanity.com/paper/2605.16649/) (2026)
- [PhotoQuilt: Training-Free Arbitrary-Resolution Photomosaics via Bootstrapped Tiled Denoising](https://mlsanity.com/paper/2606.30968/) (2026)
- [Tuning-Free Latent Diffusion Models for Ultrahigh-Resolution Image Editing](https://mlsanity.com/paper/2607.06136/) (2026)
- [HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models](https://mlsanity.com/paper/2608.07003/) (2026)
- [Backend-Agnostic Sparse Attention for Fast High-Resolution Visual Generation](https://mlsanity.com/paper/2610.08772/) (2026)
- [AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers](https://mlsanity.com/paper/2610.17227/) (2026)

## Abstract (arXiv)

Diffusion models have become a mainstream approach for high-resolution image synthesis. However, directly generating higher-resolution images from pretrained diffusion models will encounter unreasonable object duplication and exponentially increase the generation time. In this paper, we discover that object duplication arises from feature duplication in the deep blocks of the U-Net. Concurrently, We pinpoint the extended generation times to self-attention redundancy in U-Net's top blocks. To address these issues, we propose a tuning-free higher-resolution framework named HiDiffusion. Specifically, HiDiffusion contains Resolution-Aware U-Net (RAU-Net) that dynamically adjusts the feature map size to resolve object duplication and engages Modified Shifted Window Multi-head Self-Attention (MSW-MSA) that utilizes optimized window attention to reduce computations. we can integrate HiDiffusion into various pretrained diffusion models to scale image generation resolutions even to 4096x4096 at 1.5-6x the inference speed of previous methods. Extensive experiments demonstrate that our approach can address object duplication and heavy computation issues, achieving state-of-the-art performance on higher-resolution image synthesis tasks.