# Second Order Drifting Models

arXiv: https://arxiv.org/abs/2608.07924 · HTML: https://mlsanity.com/paper/2608.07924/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 55 references that are in the map.
- First posted on arXiv 8 Aug 2026.

## Builds on (most cited first, 30 of 38)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [Generative Modeling via Drifting](https://mlsanity.com/paper/2602.04770/) (2026)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [How to build a consistency model: Learning flow maps via self-distillation](https://mlsanity.com/paper/2505.18825/) (2025)
- [On Second Order Behaviour in Augmented Neural ODEs](https://mlsanity.com/paper/2006.07220/) (2020)
- [Heavy Ball Neural Ordinary Differential Equations](https://mlsanity.com/paper/2110.04840/) (2021)
- [Flood Prediction Using Machine Learning Models: Literature Review](https://mlsanity.com/paper/1908.02781/) (2019)
- [Variational Rectified Flow Matching](https://mlsanity.com/paper/2502.09616/) (2025)
- [Continuous Time Analysis of Momentum Methods](https://mlsanity.com/paper/1906.04285/) (2019)
- [User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems](https://mlsanity.com/paper/2306.07526/) (2023)
- [Acceleration via Symplectic Discretization of High-Resolution Differential Equations](https://mlsanity.com/paper/1902.03694/) (2019)
- [Sinkhorn-Drifting Generative Models](https://mlsanity.com/paper/2603.12366/) (2026)
- [Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective](https://mlsanity.com/paper/2603.09936/) (2026)
- [MomentumRNN: Integrating Momentum into Recurrent Neural Networks](https://mlsanity.com/paper/2006.06919/) (2020)
- [Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences](https://mlsanity.com/paper/2603.10592/) (2026)
- [Fast convex optimization via inertial dynamics with Hessian driven damping](https://mlsanity.com/paper/1601.07113/) (2016)

## Built on by (most cited first, 1 of 1)

- [Refresh or Realize? Compute Allocation in Drifting Models](https://mlsanity.com/paper/2609.32298/) (2026)

## Abstract (arXiv)

Drifting models are a recent class of one-step generative models that evolve the model distribution during training using a predefined sample-based drift field. Although they avoid iterative inference, their kernel-based drift fields induce frequency-dependent training dynamics: In the linearized regime, each Fourier mode of the density residual decays at a rate determined by the kernel spectrum, leading to slow recovery of fine-scale structure. We propose Second-Order Drifting Models, which lift drifting dynamics into phase space by augmenting generated samples with artificial velocity variables. We show that the resulting density perturbations obey accelerated second-order dynamics in Fourier space, connecting drifting models to the celebrated Nesterov acceleration from optimization theory. This provides a principled mechanism for mitigating the spectral stiffness of first-order drifting while preserving one-step inference. We derive a practical semi-implicit training algorithm and evaluate it on synthetic distribution matching, sequential data generation, and robotic control. Across these settings, the second-order drifting model improves convergence behavior and achieves competitive or superior performance over first-order drifting baselines.