# ReDi: Rectified Discrete Flow

arXiv: https://arxiv.org/abs/2507.15897 · HTML: https://mlsanity.com/paper/2507.15897/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024)
- [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [MAGVIT: Masked Generative Video Transformer](https://mlsanity.com/paper/2212.05199/) (2022)
- [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024)
- [Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling](https://mlsanity.com/paper/2409.02908/) (2024)
- [Simple Guidance Mechanisms for Discrete Diffusion Models](https://mlsanity.com/paper/2412.10193/) (2024)
- [The Diffusion Duality](https://mlsanity.com/paper/2506.10892/) (2025)
- [MaskBit: Embedding-free Image Generation via Bit Tokens](https://mlsanity.com/paper/2409.16211/) (2024)
- [PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Accelerator](https://mlsanity.com/paper/2405.07510/) (2024)
- [Beyond Autoregression: Fast LLMs via Self-Distillation Through Time](https://mlsanity.com/paper/2410.21035/) (2024)
- [Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning](https://mlsanity.com/paper/2312.11420/) (2023)
- [Distillation of Discrete Diffusion through Dimensional Correlations](https://mlsanity.com/paper/2410.08709/) (2024)
- [Simple ReFlow: Improved Techniques for Fast Flow Models](https://mlsanity.com/paper/2410.07815/) (2024)
- [A Pytorch Reproduction of Masked Generative Image Transformer](https://mlsanity.com/paper/2310.14400/) (2023)
- [Discrete Diffusion Schr"odinger Bridge Matching for Graph Transformation](https://mlsanity.com/paper/2410.01500/) (2024)
- [Analyzing and Mitigating Model Collapse in Rectified Flow Models](https://mlsanity.com/paper/2412.08175/) (2024)
- [Towards End-to-End Generative Modeling of Long Videos with Memory-Efficient Bidirectional Transformers](https://mlsanity.com/paper/2303.11251/) (2023)
- [Categorical Schr"odinger Bridge Matching](https://mlsanity.com/paper/2502.01416/) (2025)

## Built on by (most cited first, 14 of 14)

- [Scaling Categorical Flow Maps](https://mlsanity.com/paper/2605.07820/) (2026)
- [Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings](https://mlsanity.com/paper/2509.22925/) (2025)
- [AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance](https://mlsanity.com/paper/2510.00352/) (2025)
- [PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models](https://mlsanity.com/paper/2512.20063/) (2025)
- [S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation](https://mlsanity.com/paper/2603.25702/) (2026)
- [ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusion](https://mlsanity.com/paper/2604.09450/) (2026)
- [Few-Step Diffusion Language Models via Trajectory Self-Distillation](https://mlsanity.com/paper/2602.12262/) (2026)
- [Coupling Models for One-Step Discrete Generation](https://mlsanity.com/paper/2605.07193/) (2026)
- [Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment](https://mlsanity.com/paper/2607.04311/) (2026)
- [One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration](https://mlsanity.com/paper/2605.21484/) (2026)
- [From Truncation to Commitment: Persistent Context in Uniform Discrete Diffusion](https://mlsanity.com/paper/2609.01043/) (2026)
- [E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models](https://mlsanity.com/paper/2609.37533/) (2026)
- [Distribution Matching Distillation for Continuous Diffusion Language Models](https://mlsanity.com/paper/2609.40235/) (2026)
- [ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation](https://mlsanity.com/paper/2610.13225/) (2026)

## Abstract (arXiv)

Discrete Flow-based Models (DFMs) are powerful generative models for high-quality discrete data but typically suffer from slow sampling speeds due to their reliance on iterative decoding processes. This reliance on a multi-step process originates from the factorization approximation of DFMs, which is necessary for handling high-dimensional data. In this paper, we analyze the factorization approximation error using Conditional Total Correlation (TC), and reveal its dependence on the coupling. To address the challenge of efficient few-step generation, we propose Rectified Discrete Flow (ReDi), a novel iterative method that reduces the underlying factorization error (measured as Conditional TC) by rectifying the coupling between source and target distributions. We theoretically prove that each ReDi step guarantees a monotonic decreasing Conditional TC, ensuring its convergence. Empirically, ReDi significantly reduces Conditional TC and enables few-step generation. Moreover, we demonstrate that the rectified couplings are well-suited for training efficient one-step models on image generation. ReDi offers a simple and theoretically grounded approach for tackling the few-step challenge, providing a new perspective on efficient discrete data synthesis. Code is available at https://github.com/Ugness/ReDi_discrete.