mlsanityData to 7 Oct 2026

Lagrangian--Hamiltonian Flows for Video Prediction and Image Generation: A Symplectic Perspective

arXiv 2609.35710 ↗First posted 28 Sep 2026cs.CVcited by 0

On the map: Diffusion models → Video generation also fits: Flow matching, time series and science

Abstract (arXiv)

We introduce LHFM, a geometric framework for learning image dynamics. Drawing on structures central to classical mechanics, symplectic geometry, and geometric quantization, LHFM represents each image as an exact Lagrangian graph and models its evolution through image-dependent Hamiltonian flows, which yield a transport--source parameterization of image velocities. Our primary application is deterministic video prediction: LHFM-V is a recurrent model that advances frames by integrating predicted transport and source fields, and achieves the lowest reported FLOP count among the compared recurrent models with similar prediction accuracy. The image variant, LHFM-I, shows that the same construction is compatible with flow matching: in a matched experiment, it attains a lower FID than the flow-matching baseline.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 16 of its 29 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

16 of 29 references.

References · 16

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  5. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  6. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  7. 2023Flow Matching on General Geometriescited by 260
  8. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
  9. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  10. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  11. 2018PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learningcited by 122
  12. 2018Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamicscited by 91
4 more
  1. 2024Metric Flow Matching for Smooth Interpolations on the Data Manifoldcited by 74
  2. 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 54
  3. 2023SwinLSTM:Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTMcited by 32
  4. 2024Video Prediction Transformers without Recurrence or Convolutioncited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lagrangian Flow Matching: A Least-Action Framework for Principled Path Design
  2. 2026Eulerian Motion Guidance: Robust Image Animation via Bidirectional Geometric Consistency
  3. 2026Constrained Flow Matching via Lagrangian Dual Flows
  4. 2026A Lagrangian View of Flow Matching
  5. 2025Purrception: Variational Flow Matching for Vector-Quantized Image Generation
  6. 2026StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
  7. 2026Unlocking Temporal Generalization in Hamiltonian Video Dynamics Models
  8. 2025Graph Flow Matching: Enhancing Image Generation with Neighbor-Aware Flow Fields