# Neural Differential Appearance Equations

arXiv: https://arxiv.org/abs/2410.07128 · HTML: https://mlsanity.com/paper/2410.07128/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 81 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Single-Image SVBRDF Capture with a Rendering-Aware Deep Network](https://mlsanity.com/paper/1810.09718/) (2018)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [SinFusion: Training Diffusion Models on a Single Image or Video](https://mlsanity.com/paper/2211.11743/) (2022)
- [Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks](https://mlsanity.com/paper/1809.00886/) (2018)
- [Iterative α-(de)Blending: a Minimalist Deterministic Diffusion Model](https://mlsanity.com/paper/2305.03486/) (2023)
- [Generative Modelling of BRDF Textures from Flash Images](https://mlsanity.com/paper/2102.11861/) (2021)
- [MaterialGAN: Reflectance Capture using a Generative SVBRDF Model](https://mlsanity.com/paper/2010.00114/) (2020)
- [MatFusion: A Generative Diffusion Model for SVBRDF Capture](https://mlsanity.com/paper/2406.06539/) (2024)
- [DyNCA: Real-time Dynamic Texture Synthesis Using Neural Cellular Automata](https://mlsanity.com/paper/2211.11417/) (2022)
- [Two-Stream Convolutional Networks for Dynamic Texture Synthesis](https://mlsanity.com/paper/1706.06982/) (2017)
- [Mesh Neural Cellular Automata](https://mlsanity.com/paper/2311.02820/) (2023)
- [Texture Generation with Neural Cellular Automata](https://mlsanity.com/paper/2105.07299/) (2021)
- [Synthesising Dynamic Textures using Convolutional Neural Networks](https://mlsanity.com/paper/1702.07006/) (2017)
- [Dynamic Texture Synthesis by Incorporating Long-range Spatial and Temporal Correlations](https://mlsanity.com/paper/2104.05940/) (2021)

## Built on by (most cited first, 1 of 1)

- [Generative Video Bi-flow](https://mlsanity.com/paper/2503.06364/) (2025)

## Abstract (arXiv)

We propose a method to reproduce dynamic appearance textures with space-stationary but time-varying visual statistics. While most previous work decomposes dynamic textures into static appearance and motion, we focus on dynamic appearance that results not from motion but variations of fundamental properties, such as rusting, decaying, melting, and weathering. To this end, we adopt the neural ordinary differential equation (ODE) to learn the underlying dynamics of appearance from a target exemplar. We simulate the ODE in two phases. At the "warm-up" phase, the ODE diffuses a random noise to an initial state. We then constrain the further evolution of this ODE to replicate the evolution of visual feature statistics in the exemplar during the generation phase. The particular innovation of this work is the neural ODE achieving both denoising and evolution for dynamics synthesis, with a proposed temporal training scheme. We study both relightable (BRDF) and non-relightable (RGB) appearance models. For both we introduce new pilot datasets, allowing, for the first time, to study such phenomena: For RGB we provide 22 dynamic textures acquired from free online sources; For BRDFs, we further acquire a dataset of 21 flash-lit videos of time-varying materials, enabled by a simple-to-construct setup. Our experiments show that our method consistently yields realistic and coherent results, whereas prior works falter under pronounced temporal appearance variations. A user study confirms our approach is preferred to previous work for such exemplars.