# CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Models

arXiv: https://arxiv.org/abs/2509.24526 · HTML: https://mlsanity.com/paper/2509.24526/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 49 of its 57 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 49)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [Analyzing and Improving the Training Dynamics of Diffusion Models](https://mlsanity.com/paper/2312.02696/) (2023)
- [One Step Diffusion via Shortcut Models](https://mlsanity.com/paper/2410.12557/) (2024)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)

## Built on by (most cited first, 18 of 18)

- [A Unified View of Score-Based and Drifting Models](https://mlsanity.com/paper/2603.07514/) (2026)
- [Parallel Decoding Distillation for Fast Image and Video Generation](https://mlsanity.com/paper/2607.26004/) (2026)
- [Learning Straight Flows: Variational Flow Matching for Efficient Generation](https://mlsanity.com/paper/2511.17583/) (2025)
- [MeanFlow Transformers with Representation Autoencoders](https://mlsanity.com/paper/2511.13019/) (2025)
- [Understanding, Accelerating, and Improving MeanFlow Training](https://mlsanity.com/paper/2511.19065/) (2025)
- [Midpoint Generative Models](https://mlsanity.com/paper/2605.29920/) (2026)
- [Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control](https://mlsanity.com/paper/2609.17909/) (2026)
- [Self-Evaluation Unlocks Any-Step Text-to-Image Generation](https://mlsanity.com/paper/2512.22374/) (2025)
- [Few-Step Diffusion Language Models via Trajectory Self-Distillation](https://mlsanity.com/paper/2602.12262/) (2026)
- [Fast, faithful and photorealistic diffusion-based image super-resolution with enhanced Flow Map models](https://mlsanity.com/paper/2601.16660/) (2026)
- [Transition Flow Matching](https://mlsanity.com/paper/2603.15689/) (2026)
- [Drift Flow Matching](https://mlsanity.com/paper/2605.17244/) (2026)
- [Efficient Image Synthesis with Sphere Latent Encoder](https://mlsanity.com/paper/2605.15592/) (2026)
- [Stabilizing, Scaling & Enhancing MeanFlow for Large-scale Diffusion Distillation](https://mlsanity.com/paper/2605.17834/) (2026)
- [Overcoming the Curvature Bottleneck in MeanFlow](https://mlsanity.com/paper/2511.23342/) (2025)
- [Dual-End Consistency Model](https://mlsanity.com/paper/2602.10764/) (2026)
- [Learning Sampled-data Control for Swarms via MeanFlow](https://mlsanity.com/paper/2603.20189/) (2026)
- [Discrete Meanflow Training Curriculum](https://mlsanity.com/paper/2604.08837/) (2026)

## Abstract (arXiv)

Flow map models such as Consistency Models (CM) and Mean Flow (MF) enable few-step generation by learning the long jump of the ODE solution of diffusion models, yet training remains unstable, sensitive to hyperparameters, and costly. Initializing from a pre-trained diffusion model helps, but still requires converting infinitesimal steps into a long-jump map, leaving instability unresolved. We introduce mid-training, the first concept and practical method that inserts a lightweight intermediate stage between the (diffusion) pre-training and the final flow map training (i.e., post-training) for vision generation. Concretely, Consistency Mid-Training (CMT) is a compact and principled stage that trains a model to map points along a solver trajectory from a pre-trained model, starting from a prior sample, directly to the solver-generated clean sample. It yields a trajectory-consistent and stable initialization. This initializer outperforms random and diffusion-based baselines and enables fast, robust convergence without heuristics. Initializing post-training with CMT weights further simplifies flow map learning. Empirically, CMT achieves state of the art two step FIDs: 1.97 on CIFAR-10, 1.32 on ImageNet 64x64, and 1.84 on ImageNet 512x512, while using up to 98% less training data and GPU time, compared to CMs. On ImageNet 256x256, CMT reaches 1-step FID 3.34 while cutting total training time by about 50% compared to MF from scratch (FID 3.43). This establishes CMT as a principled, efficient, and general framework for training flow map models.