# ReCaVSR: One-Step Streaming Diffusion Video Super-Resolution with Recycled Latents and Learned Cache Routing

arXiv: https://arxiv.org/abs/2609.37831 · HTML: https://mlsanity.com/paper/2609.37831/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 35 of its 38 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)
- Efficient LLM inference and serving / [Fast autoregressive visual generation](https://mlsanity.com/inference/fast-autoregressive-visual-generation/)

## Builds on (most cited first, 30 of 35)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Diffusion Adversarial Post-Training for One-Step Video Generation](https://mlsanity.com/paper/2501.08316/) (2025)
- [Real-Time Video Generation with Pyramid Attention Broadcast](https://mlsanity.com/paper/2408.12588/) (2024)
- [Investigating Tradeoffs in Real-World Video Super-Resolution](https://mlsanity.com/paper/2111.12704/) (2021)
- [Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution](https://mlsanity.com/paper/2312.06640/) (2023)
- [FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality](https://mlsanity.com/paper/2410.19355/) (2024)
- [VSA: Faster Video Diffusion with Trainable Sparse Attention](https://mlsanity.com/paper/2505.13389/) (2025)
- [Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2506.09350/) (2025)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention](https://mlsanity.com/paper/2509.24006/) (2025)
- [STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution](https://mlsanity.com/paper/2501.02976/) (2025)
- [End-to-End Training for Autoregressive Video Diffusion via Self-Resampling](https://mlsanity.com/paper/2512.15702/) (2025)
- [SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training](https://mlsanity.com/paper/2506.05301/) (2025)
- [FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolution](https://mlsanity.com/paper/2510.12747/) (2025)
- [RealViformer: Investigating Attention for Real-World Video Super-Resolution](https://mlsanity.com/paper/2407.13987/) (2024)
- [DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolution](https://mlsanity.com/paper/2505.16239/) (2025)
- [VideoGigaGAN: Towards Detail-rich Video Super-Resolution](https://mlsanity.com/paper/2404.12388/) (2024)
- [DiffVSR: Revealing an Effective Recipe for Taming Robust Video Super-Resolution Against Complex Degradations](https://mlsanity.com/paper/2501.10110/) (2025)
- [Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restoration](https://mlsanity.com/paper/2508.14483/) (2025)
- [InfVSR: Toward Consistency-Driven Streaming Generative Video Super-Resolution](https://mlsanity.com/paper/2510.00948/) (2025)
- [Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion](https://mlsanity.com/paper/2512.23709/) (2025)
- [Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution](https://mlsanity.com/paper/2603.00458/) (2026)

## Abstract (arXiv)

Real-time diffusion-based video super-resolution (VSR) is in high demand for online streaming, yet stringent latency requirements often compromise generative fidelity. We propose ReCaVSR, a Wan2.2-based, one-step framework for streaming VSR that builds on two observations: recycled SR latents retain local temporal context, reducing the need for full historical Key-Value (KV) caches; and individual transformer layers benefit from distinct temporal scopes. ReCaVSR combines three complementary designs: (i) layer-wise cache routing with recycled SR latents: each DiT layer learns its KV-cache temporal scope under a cache budget and exports a static inference schedule, while recycled SR latents propagate local context by conditioning each new block on the model's own preceding predictions. (ii) Multi-Scope Query (MSQ) Discriminator: a compositional discriminator combining global, spatial-window, and temporal-tube feedback for holistic realism, local texture generation, and temporal stability. (iii) LR-conditioned adaptation of FlashDecoder: a VAE decoder that incorporates LR observations for efficient latent decoding. ReCaVSR enables streaming VSR without iterative sampling or full historical KV-cache materialization. Experiments on synthetic and real-world VSR benchmarks show better perceptual quality, temporal consistency, and streaming efficiency than representative VSR baselines. At 1080×1920 output resolution on a single NVIDIA A100-80GB, ReCaVSR achieves 21.20 FPS with 15.16 GB peak allocated GPU memory, running 2.72× faster while using 38.0% less peak allocated memory than FlashVSR Tiny. The code is available at https://github.com/kopperx/ReCaVSR.