# Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis

arXiv: https://arxiv.org/abs/2306.08645 · HTML: https://mlsanity.com/paper/2306.08645/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 56 of its 61 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 56)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)

## Built on by (most cited first, 30 of 56)

- [Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers](https://mlsanity.com/paper/2405.05945/) (2024)
- [ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models](https://mlsanity.com/paper/2310.07702/) (2023)
- [UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks](https://mlsanity.com/paper/2407.02158/) (2024)
- [FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.12963/) (2024)
- [Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptation](https://mlsanity.com/paper/2402.10491/) (2024)
- [Boosting Latent Diffusion with Flow Matching](https://mlsanity.com/paper/2312.07360/) (2023)
- [ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation](https://mlsanity.com/paper/2311.18822/) (2023)
- [DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance](https://mlsanity.com/paper/2406.18459/) (2024)
- [AccDiffusion: An Accurate Method for Higher-Resolution Image Generation](https://mlsanity.com/paper/2407.10738/) (2024)
- [MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model](https://mlsanity.com/paper/2411.16157/) (2024)
- [FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusion](https://mlsanity.com/paper/2412.09626/) (2024)
- [HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models](https://mlsanity.com/paper/2311.17528/) (2023)
- [CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up](https://mlsanity.com/paper/2412.16112/) (2024)
- [DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes](https://mlsanity.com/paper/2412.11100/) (2024)
- [I-Max: Maximize the Resolution Potential of Pre-trained Rectified Flow Transformers with Projected Flow](https://mlsanity.com/paper/2410.07536/) (2024)
- [Collaborative Control for Geometry-Conditioned PBR Image Generation](https://mlsanity.com/paper/2402.05919/) (2024)
- [HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance](https://mlsanity.com/paper/2504.06232/) (2025)
- [UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset](https://mlsanity.com/paper/2510.20661/) (2025)
- [VGG-T3: Offline Feed-Forward 3D Reconstruction at Scale](https://mlsanity.com/paper/2602.23361/) (2026)
- [FreCaS: Efficient Higher-Resolution Image Generation via Frequency-aware Cascaded Sampling](https://mlsanity.com/paper/2410.18410/) (2024)
- [HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts](https://mlsanity.com/paper/2409.02919/) (2024)
- [FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Model](https://mlsanity.com/paper/2410.13925/) (2024)
- [Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Models](https://mlsanity.com/paper/2406.07251/) (2024)
- [DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion](https://mlsanity.com/paper/2510.20766/) (2025)
- [Curved Diffusion: A Generative Model With Optical Geometry Control](https://mlsanity.com/paper/2311.17609/) (2023)
- [CutDiffusion: A Simple, Fast, Cheap, and Strong Diffusion Extrapolation Method](https://mlsanity.com/paper/2404.15141/) (2024)
- [UltraImage: Rethinking Resolution Extrapolation in Image Diffusion Transformers](https://mlsanity.com/paper/2512.04504/) (2025)
- [iSeg: An Iterative Refinement-based Framework for Training-free Segmentation](https://mlsanity.com/paper/2409.03209/) (2024)
- [Merging and Splitting Diffusion Paths for Semantically Coherent Panoramas](https://mlsanity.com/paper/2408.15660/) (2024)
- [FreePCA: Integrating Consistency Information across Long-short Frames in Training-free Long Video Generation via Principal Component Analysis](https://mlsanity.com/paper/2505.01172/) (2025)

## Abstract (arXiv)

Diffusion models (DMs) have recently gained attention with state-of-the-art performance in text-to-image synthesis. Abiding by the tradition in deep learning, DMs are trained and evaluated on the images with fixed sizes. However, users are demanding for various images with specific sizes and various aspect ratio. This paper focuses on adapting text-to-image diffusion models to handle such variety while maintaining visual fidelity. First we observe that, during the synthesis, lower resolution images suffer from incomplete object portrayal, while higher resolution images exhibit repetitively disordered presentation. Next, we establish a statistical relationship indicating that attention entropy changes with token quantity, suggesting that models aggregate spatial information in proportion to image resolution. The subsequent interpretation on our observations is that objects are incompletely depicted due to limited spatial information for low resolutions, while repetitively disorganized presentation arises from redundant spatial information for high resolutions. From this perspective, we propose a scaling factor to alleviate the change of attention entropy and mitigate the defective pattern observed. Extensive experimental results validate the efficacy of the proposed scaling factor, enabling models to achieve better visual effects, image quality, and text alignment. Notably, these improvements are achieved without additional training or fine-tuning techniques.