# Boosting Resolution Generalization of Diffusion Transformers with Randomized Positional Encodings arXiv: https://arxiv.org/abs/2503.18719 ยท HTML: https://mlsanity.com/paper/2503.18719/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 30 of its 35 references that are in the map. - First posted on arXiv 24 Mar 2025. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021) - [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016) - [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020) - [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021) - [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019) - [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023) - [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024) - [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023) - [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023) - [Generating Images with Sparse Representations](https://mlsanity.com/paper/2103.03841/) (2021) - [Neighborhood Attention Transformer](https://mlsanity.com/paper/2204.07143/) (2022) - [Understanding and Improving Layer Normalization](https://mlsanity.com/paper/1911.07013/) (2019) - [Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT](https://mlsanity.com/paper/2406.18583/) (2024) - [ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models](https://mlsanity.com/paper/2310.07702/) (2023) - [Randomized Positional Encodings Boost Length Generalization of Transformers](https://mlsanity.com/paper/2305.16843/) (2023) - [DemoFusion: Democratising High-Resolution Image Generation With No ](https://mlsanity.com/paper/2311.16973/) (2023) - [FiT: Flexible Vision Transformer for Diffusion Model](https://mlsanity.com/paper/2402.12376/) (2024) - [GenTron: Diffusion Transformers for Image and Video Generation](https://mlsanity.com/paper/2312.04557/) (2023) - [Boundless: Generative Adversarial Networks for Image Extension](https://mlsanity.com/paper/1908.07007/) (2019) - [Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis](https://mlsanity.com/paper/2306.08645/) (2023) - [Very Long Natural Scenery Image Prediction by Outpainting](https://mlsanity.com/paper/1912.12688/) (2019) ## Built on by (most cited first, 4 of 4) - [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025) - [DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion](https://mlsanity.com/paper/2510.20766/) (2025) - [SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers](https://mlsanity.com/paper/2605.22668/) (2026) - [HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models](https://mlsanity.com/paper/2608.07003/) (2026) ## Abstract (arXiv) Resolution generalization in image generation tasks enables the production of higher-resolution images with lower training resolution overhead. However, a key obstacle for diffusion transformers in addressing this problem is the mismatch between positional encodings seen at inference and those used during training. Existing strategies such as positional encodings interpolation, extrapolation, or hybrids, do not fully resolve this mismatch. In this paper, we propose a novel two-dimensional randomized positional encodings, namely RPE-2D, that prioritizes the order of image patches rather than their absolute distances, enabling seamless high- and low-resolution generation without training on multiple resolutions. Concretely, RPE-2D independently samples positions along the horizontal and vertical axes over an expanded range during training, ensuring that the encodings used at inference lie within the training distribution and thereby improving resolution generalization. We further introduce a simple random resize-and-crop augmentation to strengthen order modeling and add micro-conditioning to indicate the applied cropping pattern. On the ImageNet dataset, RPE-2D achieves state-of-the-art resolution generalization performance, outperforming competitive methods when trained at 2562 and evaluated at 3842 and 5122, and when trained at 5122 and evaluated at 7682 and 10242. RPE-2D also exhibits outstanding capabilities in low-resolution image generation, multi-stage training acceleration, and multi-resolution inheritance.