# OmniVR: Audio-Video Conditional Generation for Archival Footage Restoration

arXiv: https://arxiv.org/abs/2608.04224 · HTML: https://mlsanity.com/paper/2608.04224/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 50 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 10 of 10)

- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [Improved Mean Flows: On the Challenges of Fastforward Generative Models](https://mlsanity.com/paper/2512.02012/) (2025)
- [VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2411.13503/) (2024)
- [AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models](https://mlsanity.com/paper/2605.24652/) (2026)
- [Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions](https://mlsanity.com/paper/2606.09150/) (2026)
- [AV-SyncBench: Decoupled Benchmarking of Temporal and Semantic Audio-Visual Synchronization](https://mlsanity.com/paper/2607.00726/) (2026)

## Built on by (most cited first, 1 of 1)

- [Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds](https://mlsanity.com/paper/2608.23383/) (2026)

## Abstract (arXiv)

Archival footage often suffers from coupled visual and acoustic degradations, yet most restoration systems process the two modalities separately. To address this problem, we present OmniVR, the first systematic framework for joint audio-video restoration, covering data construction, model adaptation, efficient inference, and evaluation. We construct a high-quality audio-video corpus with detailed captions and use a joint degradation pipeline to produce aligned clean and degraded pairs. Using these pairs, we adapt a pretrained text-to-audio-video model (T2AV) by introducing degraded audio-video conditions (TAV2AV), then progressively replace sample captions with a fixed restoration prompt while retaining caption/null rehearsal. The resulting AV2AV model requires no user-provided text. Under a compatible residual-learning model, we prove that this condition-annealing schedule reduces gradient variance and expected restoration risk relative to direct fixed-prompt adaptation at the same training budget. For efficient deployment, OmniVR-Flash combines reduced-resolution video conditioning, MeanFlow-based one-step distillation, and Turbo VAE, achieving approximately 38 fps at 1K and 18 fps at 2K on a single B200 GPU. We further introduce OmniVRBench to evaluate four complementary dimensions: visual quality, audio quality, temporal consistency, and audio-visual synchrony. OmniVR achieves state-of-the-art results on public benchmarks and OmniVRBench. Data, code, and model weights will be released. Project Page: https://xin1u.github.io/OminiVR_PAGE/