# RFLAV: Rolling Flow matching for infinite Audio Video generation

arXiv: https://arxiv.org/abs/2503.08307 · HTML: https://mlsanity.com/paper/2503.08307/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 28 of its 32 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 28 of 28)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer](https://mlsanity.com/paper/2204.03638/) (2022)
- [MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation](https://mlsanity.com/paper/2212.09478/) (2022)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [Scaling Autoregressive Video Models](https://mlsanity.com/paper/1906.02634/) (2019)
- [Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model](https://mlsanity.com/paper/2304.13731/) (2023)
- [Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners](https://mlsanity.com/paper/2402.17723/) (2024)
- [Rolling Diffusion Models](https://mlsanity.com/paper/2402.09470/) (2024)
- [ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Models](https://mlsanity.com/paper/2311.18834/) (2023)
- [Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation](https://mlsanity.com/paper/2401.01044/) (2024)
- [AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation](https://mlsanity.com/paper/2406.07686/) (2024)
- [Autoregressive Diffusion Transformer for Text-to-Speech Synthesis](https://mlsanity.com/paper/2406.05551/) (2024)
- [EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer](https://mlsanity.com/paper/2409.10819/) (2024)
- [AADiff: Audio-Aligned Video Synthesis with Text-to-Image Diffusion](https://mlsanity.com/paper/2305.04001/) (2023)
- [A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation](https://mlsanity.com/paper/2405.13762/) (2024)
- [CMMD: Contrastive Multi-Modal Diffusion for Video-Audio Conditional Modeling](https://mlsanity.com/paper/2312.05412/) (2023)

## Built on by (most cited first, 6 of 6)

- [UniVerse-1: Unified Audio-Video Generation via Stitching of Experts](https://mlsanity.com/paper/2509.06155/) (2025)
- [LD-LAudio-V1: Video-to-Long-Form-Audio Generation Extension with Dual Lightweight Adapters](https://mlsanity.com/paper/2508.11074/) (2025)
- [Autoregressive One-Step Generative Modeling for Dynamical System Forecasting](https://mlsanity.com/paper/2605.05540/) (2026)
- [FacePlex: Toward Natural Full-Duplex Conversational Avatars](https://mlsanity.com/paper/2606.30145/) (2026)
- [Encore: Infinite Audio-Video Generation with Adaptive Signal Routing](https://mlsanity.com/paper/2609.04249/) (2026)
- [Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy](https://mlsanity.com/paper/2609.34381/) (2026)

## Abstract (arXiv)

Joint audio-video (AV) generation is still a significant challenge in generative AI, primarily due to three critical requirements: quality of the generated samples, seamless multimodal synchronization and temporal coherence, with audio tracks that match the visual data and vice versa, and limitless video duration. In this paper, we present R-FLAV, a novel transformer-based architecture that addresses all the key challenges of AV generation. We explore three distinct cross modality interaction modules, with our lightweight temporal fusion module emerging as the most effective and computationally efficient approach for aligning audio and visual modalities. Our experimental results demonstrate that R-FLAV outperforms existing state-of-the-art models in multimodal AV generation tasks. Our code and checkpoints are available at https://github.com/ErgastiAlex/R-FLAV.