mlsanityData to 7 Oct 2026

Efficient Video Prediction via Sparsely Conditioned Flow Matching

arXiv 2211.14575 ↗First posted 26 Nov 2022cs.CVcited by 54

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

We introduce a novel generative model for video prediction based on latent flow matching, an efficient alternative to diffusion-based models. In contrast to prior work, we keep the high costs of modeling the past during training and inference at bay by conditioning only on a small random set of past frames at each integration step of the image generation process. Moreover, to enable the generation of high-resolution videos and to speed up the training, we work in the latent space of a pretrained VQGAN. Finally, we propose to approximate the initial condition of the flow ODE with the previous noisy frame. This allows to reduce the number of integration steps and hence, speed up the sampling at inference time. We call our model Random frame conditioned flow Integration for VidEo pRediction, or, in short, RIVER. We show that RIVER achieves superior or on par performance compared to prior work on common video prediction benchmarks, while requiring an order of magnitude fewer computational resources.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 71 of its 84 references that are in the map. First posted on arXiv 26 Nov 2022.

Builds on

71 of 84 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Flow Matching for Generative Modeling6 Oct
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow7 Sep
  3. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation7 Jul

Other references · 68

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2016Layer Normalizationcited by 5,579
  12. 2017Neural Discrete Representation Learningcited by 4,650
38 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2022Video Diffusion Modelscited by 2,280
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  7. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  10. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  11. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  12. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  13. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  14. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  15. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  16. 2022Novel View Synthesis with Diffusion Modelscited by 299
  17. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  18. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  19. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  20. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  21. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  22. 2018Stochastic Video Generation with a Learned Priorcited by 244
  23. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  24. 2017Stochastic Variational Video Predictioncited by 234
  25. 2021Gotta Go Fast When Generating Data with Score-Based Modelscited by 218
  26. 2019Scaling Autoregressive Video Modelscited by 189
  27. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  28. 2018Stochastic Adversarial Video Predictioncited by 173
  29. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  30. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
  31. 2017Unsupervised Learning of Disentangled Representations from Videocited by 138
  32. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 137
  33. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  34. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  35. 2018PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learningcited by 122
  36. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  37. 2020Stochastic Latent Residual Video Predictioncited by 106
  38. 2020Latent Video Transformercited by 106

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 19; 2025: 18; 2026: 14.

54 papers.

Sort:

By area:

  1. 2024Riemannian Flow Matching Policy for Robot Motion Learningcited by 61
  2. 2024Probabilistic Forecasting with Stochastic Interpolants and F"ollmer Processescited by 61
  3. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  4. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  5. 2023Boosting Latent Diffusion with Flow Matchingcited by 44
  6. 2025ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Modelscited by 37
  7. 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 36
  8. 2025LBM: Latent Bridge Matching for Fast Image-to-Image Translationcited by 36
  9. 2024Dynamic Conditional Optimal Transport through Simulation-Free Flowscited by 24
  10. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  11. 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 22
  12. 2024Fast and Robust Visuomotor Riemannian Flow Matching Policycited by 18
38 more
  1. 2024AKiRa: Augmentation Kit on Rays for optical video generationcited by 17
  2. 2024Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generationcited by 14
  3. 2025FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulationcited by 13
  4. 2024Elucidating the Design Choice of Probability Paths in Flow Matching for Forecastingcited by 12
  5. 2024Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equationcited by 11
  6. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 11
  7. 2025FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generationcited by 10
  8. 2024Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph Generationcited by 9
  9. 2026Improving Flow Matching by Aligning Flow Divergencecited by 9
  10. 2024Parametric model reduction of mean-field and stochastic systems via higher-order action matchingcited by 8
  11. 2023Learn the Force We Can: Enabling Sparse Motion Control in Multi-Object Video Generationcited by 7
  12. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approachescited by 7
  13. 2025ARFlow: Human Action-Reaction Flow Matching with Physical Guidancecited by 7
  14. 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
  15. 2025Longitudinal Flow Matching for Trajectory Modelingcited by 6
  16. 2026Faster Inference of Flow-Based Generative Models via Improved Data-Noise Couplingcited by 6
  17. 2025Time-Unified Diffusion Policy with Action Discrimination for Robotic Manipulationcited by 5
  18. 2025EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observationscited by 5
  19. 2026FlowLet: Conditional 3D Brain MRI Synthesis using Wavelet Flow Matchingcited by 5
  20. 2026DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Taskscited by 5
  21. 2024ContextFlow++: Generalist-Specialist Flow-based Generative Models with Mixed-Variable Context Encodingcited by 3
  22. 2025Blockwise Flow Matching: Improving Flow Matching Models For Efficient High-Quality Generationcited by 3
  23. 2026Composition of Memory Experts for Diffusion World Modelscited by 3
  24. 2024CAGE: Unsupervised Visual Composition and Animation for Controllable Video Generationcited by 2
  25. 2025Generative Video Bi-flowcited by 2
  26. 2025TimeFlow: Towards Stochastic-Aware and Efficient Time Series Generation via Flow Matching Modelingcited by 2
  27. 2024Predicting Long-horizon Futures by Conditioning on Geometry and Timecited by 1
  28. 2024Continuous Video Process: Modeling Videos as Continuous Multi-Dimensional Processes for Video Predictioncited by 1
  29. 2025Operator Flow Matching for Timeseries Forecastingcited by 1
  30. 2026Stochastic Lifting for Generating Trajectories of Stochastic Physical Systemscited by 1
  31. 2024Video Generation with Learned Action Priorcited by 0
  32. 2024Gen-Swarms: Adapting Deep Generative Models to Swarms of Dronescited by 0
  33. 2024Efficient Continuous Video Flow Model for Video Predictioncited by 0
  34. 2025Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspectivecited by 0
  35. 2025What We Don't C: Manifold Disentanglement for Structured Discoverycited by 0
  36. 2026UniFluids: Unified Neural Operator Learning with Conditional Flow-matchingcited by 0
  37. 2026Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matchingcited by 0
  38. 2026Probabilistic Inversion with Flow Matchingcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Flowception: Temporally Expansive Flow Matching for Video Generation
  2. 2025Video Latent Flow Matching: Optimal Polynomial Projections for Video Interpolation and Extrapolation
  3. 2026FastFlow: Accelerating The Generative Flow Matching Models with Bandit Inference
  4. 2025STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows
  5. 2023Flow Matching in Latent Space
  6. 2024Discrete Flow Matching
  7. 2026Transition Matching Distillation for Fast Video Generation
  8. 2025Efficient Flow Matching using Latent Variables