# Generative Modeling via Drifting

arXiv: https://arxiv.org/abs/2602.04770 · HTML: https://mlsanity.com/paper/2602.04770/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 73 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Builds on (most cited first, 30 of 55)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)

## Built on by (most cited first, 30 of 116)

- [Representation Fr'echet Loss for Visual Generation](https://mlsanity.com/paper/2604.28190/) (2026)
- [Sinkhorn-Drifting Generative Models](https://mlsanity.com/paper/2603.12366/) (2026)
- [One-Step Generative Modeling via Wasserstein Gradient Flows](https://mlsanity.com/paper/2605.11755/) (2026)
- [Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective](https://mlsanity.com/paper/2603.09936/) (2026)
- [Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences](https://mlsanity.com/paper/2603.10592/) (2026)
- [ELT: Elastic Looped Transformers for Visual Generation](https://mlsanity.com/paper/2604.09168/) (2026)
- [Continuous Latent Diffusion Language Model](https://mlsanity.com/paper/2605.06548/) (2026)
- [Drifting Fields are not Conservative](https://mlsanity.com/paper/2604.06333/) (2026)
- [A Unified View of Score-Based and Drifting Models](https://mlsanity.com/paper/2603.07514/) (2026)
- [On the Wasserstein Gradient Flow Interpretation of Drifting Models](https://mlsanity.com/paper/2605.05118/) (2026)
- [Kernel-Gradient Drifting Models](https://mlsanity.com/paper/2605.10727/) (2026)
- [Drift-Based Policy Optimization: Native One-Step Policy Learning for Online Robot Control](https://mlsanity.com/paper/2604.03540/) (2026)
- [Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion](https://mlsanity.com/paper/2605.23346/) (2026)
- [OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation](https://mlsanity.com/paper/2606.22174/) (2026)
- [Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models](https://mlsanity.com/paper/2605.22795/) (2026)
- [Drift Q-Learning](https://mlsanity.com/paper/2606.00350/) (2026)
- [DriftWorld: Fast World Modeling through Drifting](https://mlsanity.com/paper/2607.15065/) (2026)
- [Three-Body Scattering for Generative Modeling](https://mlsanity.com/paper/2607.18198/) (2026)
- [Friction-Augmented Drifting Models for Resource-Efficient Domain Translation](https://mlsanity.com/paper/2604.18194/) (2026)
- [Speech Enhancement Based on Drifting Models](https://mlsanity.com/paper/2604.24199/) (2026)
- [Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations](https://mlsanity.com/paper/2605.07327/) (2026)
- [Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flow](https://mlsanity.com/paper/2605.07727/) (2026)
- [Geometry-Aware Discretization Error of Diffusion Models](https://mlsanity.com/paper/2605.08392/) (2026)
- [DriftXpress: Faster Drifting Models via Projected RKHS Fields](https://mlsanity.com/paper/2605.12183/) (2026)
- [Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference](https://mlsanity.com/paper/2605.26552/) (2026)
- [Drifting Preference Optimization for One-Step Generative Models](https://mlsanity.com/paper/2606.02521/) (2026)
- [Fast Organic Crystal Structure Prediction with Unit Cell Flow Matching](https://mlsanity.com/paper/2606.03199/) (2026)
- [Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models](https://mlsanity.com/paper/2606.05737/) (2026)
- [Distilling Drifting Transformers with Representation Autoencoders](https://mlsanity.com/paper/2606.15553/) (2026)
- [PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generation](https://mlsanity.com/paper/2606.19935/) (2026)

## Abstract (arXiv)

Generative modeling can be formulated as learning a mapping f such that its pushforward distribution matches the data distribution. The pushforward behavior can be carried out iteratively at inference time, for example in diffusion and flow-based models. In this paper, we propose a new paradigm called Drifting Models, which evolve the pushforward distribution during training and naturally admit one-step inference. We introduce a drifting field that governs the sample movement and achieves equilibrium when the distributions match. This leads to a training objective that allows the neural network optimizer to evolve the distribution. In experiments, our one-step generator achieves state-of-the-art results on ImageNet at 256 x 256 resolution, with an FID of 1.54 in latent space and 1.61 in pixel space. We hope that our work opens up new opportunities for high-quality one-step generation.