VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation
Not on the map. Why?
Abstract (arXiv)
Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions. However, a central challenge in video prediction is that the future is highly uncertain: a sequence of past observations of events can imply many possible futures. Although a number of recent works have studied probabilistic models that can represent uncertain futures, such models are either extremely expensive computationally as in the case of pixel-level autoregressive models, or do not directly optimize the likelihood of the data. To our knowledge, our work is the first to propose multi-frame video prediction with normalizing flows, which allows for direct optimization of the data likelihood, and produces high-quality stochastic predictions. We describe an approach for modeling the latent space dynamics, and demonstrate that flow-based generative models offer a viable and competitive approach to generative modelling of video.
Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Mar 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 26; 2022: 20; 2023: 9; 2024: 7; 2025: 7; 2026: 5.
74 papers.
Sort:
By area:
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
- 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
- 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
- 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
- 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
- 2022Generating Long Videos of Dynamic Scenescited by 113
- 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
- 2021Playable Video Generationcited by 63
38 more
- 2021Predicting Video with VQVAEcited by 61
- 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 54
- 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
- 2021Learning the Predictability of the Futurecited by 48
- 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
- 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
- 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 41
- 2021Clockwork Variational Autoencoderscited by 40
- 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
- 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
- 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
- 2023Feature-Conditioned Cascaded Video Diffusion Models for Precise Echocardiogram Synthesiscited by 32
- 2022Playable Environments: Video Manipulation in Space and Timecited by 29
- 2023The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detectorcited by 25
- 2021Transflower: probabilistic autoregressive dance generation with multimodal attentioncited by 22
- 2021VisuoSpatial Foresight for Physical Sequential Fabric Manipulationcited by 18
- 2022Target-aware Molecular Graph Generationcited by 18
- 2021Diverse Video Generation using a Gaussian Process Triggercited by 17
- 2022Invertible Rescaling Network and Its Extensionscited by 15
- 2025Synthetic Video Enhances Physical Fidelity in Video Synthesiscited by 15
- 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generationcited by 14
- 2021Learning Temporal Dynamics from Cycles in Narrated Videocited by 13
- 2021Accurate Grid Keypoint Learning for Efficient Video Predictioncited by 13
- 2022STGlow: A Flow-based Generative Framework with Dual Graphormer for Pedestrian Trajectory Predictioncited by 13
- 2023Sign Language Production with Latent Motion Transformercited by 13
- 2024Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Modelcited by 13
- 2022Towards Smooth Video Compositioncited by 12
- 2021Future Frame Prediction for Robot-assisted Surgerycited by 11
- 2021Click to Move: Controlling Video Generation with Sparse Motioncited by 11
- 2022Gradual Domain Adaptation via Normalizing Flowscited by 11
- 2021Variational Predictive Routing with Nested Subjective Timescalescited by 9
- 2021S3RP: Self-Supervised Super-Resolution and Prediction for Advection-Diffusion Processcited by 9
- 2025STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flowscited by 9
- 2021Layered Controllable Video Generationcited by 7
- 2022Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learningcited by 7
- 2022Distributional Drift Adaptation with Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecastingcited by 7
- 2023A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inferencecited by 7
- 2024Review for Handling Missing Data with special missing mechanismcited by 7
50 most cited of 74 All 74 (JSON)
- 2026SignRefine: Adapting Foundational Video Models for Sign Language Generation8 Sep 2026
- 2026Joint Flow Matching for Generator-Consistent Classification27 Jul 2026
- 2026Consistency-Preserving Diverse Video Generation17 Feb 2026
- 2026Rethinking Refinement: Correcting Generative Bias without Noise Injection29 Jan 2026
- 2026Human detectors are surprisingly powerful reward models15 Jan 2026
- 2025STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows25 Nov 2025
- 2025Physics-Guided Motion Loss for Video Generation Model2 Jun 2025
- 2025Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling7 Apr 2025
- 2025Synthetic Video Enhances Physical Fidelity in Video Synthesis26 Mar 2025
- 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistency26 Mar 2025
- 2025Other Vehicle Trajectories Are Also Needed: A Driving World Model Unifies Ego-Other Vehicle Trajectories in Video Latent Space12 Mar 2025
- 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation18 Feb 2025
38 more
- 2024Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representation31 Jul 2024
- 2024Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Model31 Jul 2024
- 2024SNED: Superposition Network Architecture Search for Efficient Video Diffusion Model31 May 2024
- 2024Review for Handling Missing Data with special missing mechanism7 Apr 2024
- 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition21 Mar 2024
- 2024On depth prediction for autonomous driving using self-supervised learning10 Mar 2024
- 2024Probabilistic Forecasting of Irregular Time Series via Conditional Flows9 Feb 2024
- 2023Sign Language Production with Latent Motion Transformer20 Dec 2023
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov 2023
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning17 Nov 2023
- 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs26 Aug 2023
- 2023Deep Generative Modeling-based Data Augmentation with Demonstration using the BFBT Benchmark Void Fraction Datasets19 Aug 2023
- 2023HIDFlowNet: A Flow-Based Deep Network for Hyperspectral Image Denoising20 Jun 2023
- 2023Feature-Conditioned Cascaded Video Diffusion Models for Precise Echocardiogram Synthesis22 Mar 2023
- 2023The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detector21 Mar 2023
- 2023A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference23 Jan 2023
- 2022Towards Smooth Video Composition14 Dec 2022
- 2022Efficient Video Prediction via Sparsely Conditioned Flow Matching26 Nov 2022
- 2022WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Prediction25 Nov 2022
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov 2022
- 2022STGlow: A Flow-based Generative Framework with Dual Graphormer for Pedestrian Trajectory Prediction21 Nov 2022
- 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov 2022
- 2022Invertible Rescaling Network and Its Extensions9 Oct 2022
- 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct 2022
- 2022Distributional Drift Adaptation with Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting1 Sep 2022
- 2022Wildfire Forecasting with Satellite Images and Deep Generative Model19 Aug 2022
- 2022Gradual Domain Adaptation via Normalizing Flows23 Jun 2022
- 2022Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learning22 Jun 2022
- 2022Forecasting of depth and ego-motion with transformers and self-supervision15 Jun 2022
- 2022Generating Long Videos of Dynamic Scenes7 Jun 2022
- 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model13 May 2022
- 2022Zero-Episode Few-Shot Contrastive Predictive Coding: Solving intelligence tests without prior training4 May 2022
- 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer7 Apr 2022
- 2022Playable Environments: Video Manipulation in Space and Time3 Mar 2022
- 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks21 Feb 2022
- 2022Target-aware Molecular Graph Generation10 Feb 2022
- 2021Layered Controllable Video Generation24 Nov 2021
- 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtion24 Nov 2021
50 newest of 74 All 74 (JSON)
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
- 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 54
- 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 41
- 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
- 2023Feature-Conditioned Cascaded Video Diffusion Models for Precise Echocardiogram Synthesiscited by 32
- 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generationcited by 14
- 2025Physics-Guided Motion Loss for Video Generation Modelcited by 1
- 2026Consistency-Preserving Diverse Video Generationcited by 1
- 2024SNED: Superposition Network Architecture Search for Efficient Video Diffusion Modelcited by 0
- 2026Joint Flow Matching for Generator-Consistent Classificationcited by 0
12 most cited of 13 All 13 (JSON)
- 2026SignRefine: Adapting Foundational Video Models for Sign Language Generation8 Sep 2026
- 2026Joint Flow Matching for Generator-Consistent Classification27 Jul 2026
- 2026Consistency-Preserving Diverse Video Generation17 Feb 2026
- 2025Physics-Guided Motion Loss for Video Generation Model2 Jun 2025
- 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation18 Feb 2025
- 2024SNED: Superposition Network Architecture Search for Efficient Video Diffusion Model31 May 2024
- 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition21 Mar 2024
- 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs26 Aug 2023
- 2023Feature-Conditioned Cascaded Video Diffusion Models for Precise Echocardiogram Synthesis22 Mar 2023
- 2022Efficient Video Prediction via Sparsely Conditioned Flow Matching26 Nov 2022
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov 2022
- 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov 2022
12 newest of 13 All 13 (JSON)
- 2026Human detectors are surprisingly powerful reward modelscited by 1
- 2026Human detectors are surprisingly powerful reward models15 Jan 2026
- 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
- 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
- 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
- 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
- 2022Generating Long Videos of Dynamic Scenescited by 113
- 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
- 2021Playable Video Generationcited by 63
- 2021Predicting Video with VQVAEcited by 61
- 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
- 2021Learning the Predictability of the Futurecited by 48
- 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
12 most cited of 58 All 58 (JSON)
- 2026Rethinking Refinement: Correcting Generative Bias without Noise Injection29 Jan 2026
- 2025STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows25 Nov 2025
- 2025Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling7 Apr 2025
- 2025Synthetic Video Enhances Physical Fidelity in Video Synthesis26 Mar 2025
- 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistency26 Mar 2025
- 2024Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representation31 Jul 2024
- 2024Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Model31 Jul 2024
- 2024Review for Handling Missing Data with special missing mechanism7 Apr 2024
- 2024On depth prediction for autonomous driving using self-supervised learning10 Mar 2024
- 2024Probabilistic Forecasting of Irregular Time Series via Conditional Flows9 Feb 2024
- 2023Sign Language Production with Latent Motion Transformer20 Dec 2023
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning17 Nov 2023
12 newest of 58 All 58 (JSON)