mlsanityData to 14 Oct 2026

DLFR-VAE: Dynamic Latent Frame Rate VAE for Video Generation

arXiv 2502.11897 ↗First posted 17 Feb 2025cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose the Dynamic Latent Frame Rate VAE (DLFR-VAE), a training-free paradigm that can make use of adaptive temporal compression in latent space. While existing video generative models apply fixed compression rates via pretrained VAE, we observe that real-world video content exhibits substantial temporal non-uniformity, with high-motion segments containing more information than static scenes. Based on this insight, DLFR-VAE dynamically adjusts the latent frame rate according to the content complexity. Specifically, DLFR-VAE comprises two core innovations: (1) A Dynamic Latent Frame Rate Scheduler that partitions videos into temporal chunks and adaptively determines optimal frame rates based on information-theoretic content complexity, and (2) A training-free adaptation mechanism that transforms pretrained VAE architectures into a dynamic VAE that can process features with variable frame rates. Our simple but effective DLFR-VAE can function as a plug-and-play module, seamlessly integrating with existing video generation models and accelerating the video generation process.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 47 of its 60 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

47 of 60 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  11. 2018SlowFast Networks for Video Recognitioncited by 1,867
  12. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
35 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  3. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  4. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  5. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  6. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  7. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
  8. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  9. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  10. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  11. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 532
  12. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  13. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  14. 2023DeepCache: Accelerating Diffusion Models for Freecited by 359
  15. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 273
  16. 2022Post-training Quantization on Diffusion Modelscited by 267
  17. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  18. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  19. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  20. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  21. 2020Temporal Pyramid Network for Action Recognitioncited by 163
  22. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  23. 2024DiTFastAttn: Attention Compression for Diffusion Transformer Modelscited by 115
  24. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91
  25. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 79
  26. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 58
  27. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuningcited by 49
  28. 2024Improved Video VAE for Latent Video Diffusion Modelcited by 48
  29. 2023Designing a Better Asymmetric VQGAN for StableDiffusioncited by 45
  30. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 42
  31. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 41
  32. 2024Large Motion Video Autoencoding with Cross-modal Video VAEcited by 19
  33. 2020NUTA: Non-uniform Temporal Aggregation for Action Recognitioncited by 6
  34. 2024Deep video representation learning: a surveycited by 4
  35. 2023SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolutioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025MotionStrata: Hierarchical Motion Latents for Compact Video Autoencodingcited by 9
  2. 2025VGDFR: Diffusion-based Video Generation with Dynamic Latent Frame Ratecited by 7
  3. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  4. 2026Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusioncited by 0