# Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter

arXiv: https://arxiv.org/abs/2508.16939 · HTML: https://mlsanity.com/paper/2508.16939/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 23 Aug 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 17 of 17)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Neural Rough Differential Equations for Long Time Series](https://mlsanity.com/paper/2009.08295/) (2020)
- [Phased Consistency Models](https://mlsanity.com/paper/2405.18407/) (2024)
- [Neural Controlled Differential Equations for Online Prediction Tasks](https://mlsanity.com/paper/2106.11028/) (2021)
- [Signature Methods in Machine Learning](https://mlsanity.com/paper/2206.14674/) (2022)
- [Learning stochastic differential equations using RNN with log signature features](https://mlsanity.com/paper/1908.08286/) (2019)
- [Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition](https://mlsanity.com/paper/2110.13008/) (2021)
- [Generative Modelling of L'evy Area for High Order SDE Simulation](https://mlsanity.com/paper/2308.02452/) (2023)

## Built on by (most cited first, 3 of 3)

- [Strong Stochastic Flow Maps](https://mlsanity.com/paper/2606.01086/) (2026)
- [UniFluids: Unified Neural Operator Learning with Conditional Flow-matching](https://mlsanity.com/paper/2603.22309/) (2026)
- [Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards](https://mlsanity.com/paper/2605.10313/) (2026)

## Abstract (arXiv)

Diffusion models achieve state-of-the-art generative performance but remain computationally expensive at inference, often requiring many discretization steps. We propose Sig-DEG (Signature-based Differential Equation Generator), a generator for distilling pre-trained diffusion models. Inspired by high-order stochastic Taylor expansions, Sig-DEG combines partial signatures that summarize Brownian motion over subintervals with a recurrent structure to approximate stochastic differential equation solutions to achieve universality. We formulate distillation as supervised learning, matching fine-resolution teacher trajectories on a coarse time grid. At inference, Sig-DEG enables fast generation, as the partial signatures can be simulated exactly from their known distributions without requiring fine-grained Brownian paths. Experiments on image and time-series generation demonstrate competitive sample quality with inference speedups of up to several hundred times over the teacher. A Schr"odinger bridge example further demonstrates Sig-DEG's applicability beyond diffusion distillation.