mlsanityData to 7 Oct 2026

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

arXiv 2403.03206 ↗First posted 5 Mar 2024cs.CVcited by 4,094

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is not yet decisively established as standard practice. In this work, we improve existing noise sampling techniques for training rectified flow models by biasing them towards perceptually relevant scales. Through a large-scale study, we demonstrate the superior performance of this approach compared to established diffusion formulations for high-resolution text-to-image synthesis. Additionally, we present a novel transformer-based architecture for text-to-image generation that uses separate weights for the two modalities and enables a bidirectional flow of information between image and text tokens, improving text comprehension, typography, and human preference ratings. We demonstrate that this architecture follows predictable scaling trends and correlates lower validation loss to improved text-to-image synthesis as measured by various metrics and human evaluations. Our largest models outperform state-of-the-art models, and we will make our experimental data, code, and model weights publicly available.

Key facts. Cited by 4,094 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 53 of its 75 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

53 of 75 references.

Closest ancestors · 17

Same direction, newest first

  1. 2023Boosting Latent Diffusion with Flow Matching12 Dec
  2. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  3. 2023Adversarial Diffusion Distillation28 Nov
  4. 2023State of the Art on Diffusion Models for Visual Computing11 Oct
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  6. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  7. 2023Common Diffusion Noise Schedules and Sample Steps are Flawed15 May
  8. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  9. 2023Minimizing Trajectory Curvature of ODE-based Generative Models27 Jan
  10. 2022Scalable Diffusion Models with Transformers19 Dec
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
5 more
  1. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  2. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  3. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2022Flow Matching for Generative Modelingcited by 5,426
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2018Neural Ordinary Differential Equationscited by 3,733
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
24 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  3. 2017Decoupled Weight Decay Regularizationcited by 1,722
  4. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2019Root Mean Square Layer Normalizationcited by 1,425
  7. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  10. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  11. 2021Scaling Vision Transformerscited by 993
  12. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  13. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  14. 2023Extracting Training Data from Diffusion Modelscited by 723
  15. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  16. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  17. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  18. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  19. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  20. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  21. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  22. 2021Projected GANs Converge Fastercited by 204
  23. 2022A Self-Supervised Descriptor for Image Copy Detectioncited by 175
  24. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Modelscited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 549; 2025: 1857; 2026: 1688.

4,094 papers.

Sort:

By area:

  1. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  5. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  6. 2025Qwen-Image Technical Reportcited by 973
  7. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  9. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  10. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  11. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  12. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
38 more
  1. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  2. 2025Mean Flows for One-step Generative Modelingcited by 524
  3. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  4. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  5. 2025VACE: All-in-One Video Creation and Editingcited by 467
  6. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  7. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  8. 2024OmniGen: Unified Image Generationcited by 388
  9. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  10. 2024One Step Diffusion via Shortcut Modelscited by 373
  11. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  12. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  13. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  14. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  15. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  16. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  17. 2024Discrete Flow Matchingcited by 301
  18. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  19. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
  20. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
  21. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  22. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  23. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  24. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  25. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  26. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  27. 2025Improving Video Generation with Human Feedbackcited by 267
  28. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  29. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
  30. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  31. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  32. 2025Transfer between Modalities with MetaQueriescited by 231
  33. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  34. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
  35. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  36. 2024FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Modelscited by 214
  37. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
  38. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212

50 most cited of 4,094 All 4,094 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis
  2. 2025LSSGen: Leveraging Latent Space Scaling in Flow and Diffusion for Efficient Text to Image Generation
  3. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
  4. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  5. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
  6. 2023Text-driven Visual Synthesis with Latent Diffusion Prior
  7. 2024EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing
  8. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization