# Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation

arXiv: https://arxiv.org/abs/2010.08188 · HTML: https://mlsanity.com/paper/2010.08188/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2020.

## Built on by (most cited first, 30 of 40)

- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer](https://mlsanity.com/paper/2204.03638/) (2022)
- [Learning Dynamic Alignment via Meta-filter for Few-shot Learning](https://mlsanity.com/paper/2103.13582/) (2021)
- [E-CIR: Event-Enhanced Continuous Intensity Recovery](https://mlsanity.com/paper/2203.01935/) (2022)
- [Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Model](https://mlsanity.com/paper/2312.08403/) (2023)
- [Markovian Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2207.05543/) (2022)
- [Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Model](https://mlsanity.com/paper/2407.21408/) (2024)
- [STDiff: Spatio-temporal Diffusion for Continuous Stochastic Video Prediction](https://mlsanity.com/paper/2312.06486/) (2023)
- [Improving Transferability for Cross-domain Trajectory Prediction via Neural Stochastic Differential Equation](https://mlsanity.com/paper/2312.15906/) (2023)
- [InfoGCN++: Learning Representation by Predicting the Future for Online Human Skeleton-based Action Recognition](https://mlsanity.com/paper/2310.10547/) (2023)
- [Modulated Neural ODEs](https://mlsanity.com/paper/2302.13262/) (2023)
- [Towards Scene Graph Anticipation](https://mlsanity.com/paper/2403.04899/) (2024)
- [HairFIT: Pose-Invariant Hairstyle Transfer via Flow-based Hair Alignment and Semantic-Region-Aware Inpainting](https://mlsanity.com/paper/2206.08585/) (2022)
- [Variational Inference for SDEs Driven by Fractional Noise](https://mlsanity.com/paper/2310.12975/) (2023)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [A unified model for continuous conditional video prediction](https://mlsanity.com/paper/2210.05810/) (2022)
- [Controllable Video Generation by Learning the Underlying Dynamical System with Neural ODE](https://mlsanity.com/paper/2303.05323/) (2023)
- [SMURF: Continuous Dynamics for Motion-Deblurring Radiance Fields](https://mlsanity.com/paper/2403.07547/) (2024)
- [DeblurSR: Event-Based Motion Deblurring Under the Spiking Representation](https://mlsanity.com/paper/2303.08977/) (2023)
- [AnyPose: Anytime 3D Human Pose Forecasting via Neural Ordinary Differential Equations](https://mlsanity.com/paper/2309.04840/) (2023)
- [Learning to Decouple Complex Systems](https://mlsanity.com/paper/2302.01581/) (2023)
- [VidStyleODE: Disentangled Video Editing via StyleGAN and NeuralODEs](https://mlsanity.com/paper/2304.06020/) (2023)
- [ODEBrain: Continuous-Time EEG Graph for Modeling Dynamic Brain Networks](https://mlsanity.com/paper/2602.23285/) (2026)
- [Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODE](https://mlsanity.com/paper/2112.10960/) (2021)
- [Fast Fourier Inception Networks for Occluded Video Prediction](https://mlsanity.com/paper/2306.10346/) (2023)
- [ICODE: Modeling Dynamical Systems with Extrinsic Input Information](https://mlsanity.com/paper/2411.13914/) (2024)
- [Learning and Current Prediction of PMSM Drive via Differential Neural Networks](https://mlsanity.com/paper/2412.09028/) (2024)
- [PointODE: Lightweight Point Cloud Learning with Neural Ordinary Differential Equations on Edge](https://mlsanity.com/paper/2506.00438/) (2025)
- [CRONOS: Continuous Time Reconstruction for 4D Medical Longitudinal Series](https://mlsanity.com/paper/2512.16577/) (2025)
- [LARNet: Latent Action Representation for Human Action Synthesis](https://mlsanity.com/paper/2110.10899/) (2021)

## Abstract (arXiv)

Video generation models often operate under the assumption of fixed frame rates, which leads to suboptimal performance when it comes to handling flexible frame rates (e.g., increasing the frame rate of the more dynamic portion of the video as well as handling missing video frames). To resolve the restricted nature of existing video generation models' ability to handle arbitrary timesteps, we propose continuous-time video generation by combining neural ODE (Vid-ODE) with pixel-level video processing techniques. Using ODE-ConvGRU as an encoder, a convolutional version of the recently proposed neural ODE, which enables us to learn continuous-time dynamics, Vid-ODE can learn the spatio-temporal dynamics of input videos of flexible frame rates. The decoder integrates the learned dynamics function to synthesize video frames at any given timesteps, where the pixel-level composition technique is used to maintain the sharpness of individual frames. With extensive experiments on four real-world video datasets, we verify that the proposed Vid-ODE outperforms state-of-the-art approaches under various video generation settings, both within the trained time range (interpolation) and beyond the range (extrapolation). To the best of our knowledge, Vid-ODE is the first work successfully performing continuous-time video generation using real-world videos.