# SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model

arXiv: https://arxiv.org/abs/2602.21818 · HTML: https://mlsanity.com/paper/2602.21818/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 30 of 37)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [SoundNet: Learning Sound Representations from Unlabeled Video](https://mlsanity.com/paper/1610.09001/) (2016)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [SkyReels-V2: Infinite-length Film Generative Model](https://mlsanity.com/paper/2504.13074/) (2025)
- [MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation](https://mlsanity.com/paper/2212.09478/) (2022)
- [Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation](https://mlsanity.com/paper/2407.05361/) (2024)
- [TransNet V2: An effective deep network architecture for fast shot transition detection](https://mlsanity.com/paper/2008.04838/) (2020)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models](https://mlsanity.com/paper/2502.01061/) (2025)
- [VSA: Faster Video Diffusion with Trainable Sparse Attention](https://mlsanity.com/paper/2505.13389/) (2025)
- [JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization](https://mlsanity.com/paper/2503.23377/) (2025)
- [Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation](https://mlsanity.com/paper/2510.01284/) (2025)
- [Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model](https://mlsanity.com/paper/2512.13507/) (2025)
- [OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation](https://mlsanity.com/paper/2412.00115/) (2024)
- [SkyReels-A2: Compose Anything in Video Diffusion Transformers](https://mlsanity.com/paper/2504.02436/) (2025)
- [Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation](https://mlsanity.com/paper/2505.22647/) (2025)
- [UniVerse-1: Unified Audio-Video Generation via Stitching of Experts](https://mlsanity.com/paper/2509.06155/) (2025)
- [AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation](https://mlsanity.com/paper/2406.07686/) (2024)
- [VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Models](https://mlsanity.com/paper/2410.00741/) (2024)
- [SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers](https://mlsanity.com/paper/2502.10841/) (2025)

## Built on by (most cited first, 12 of 12)

- [LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing](https://mlsanity.com/paper/2606.06042/) (2026)
- [CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation](https://mlsanity.com/paper/2606.09639/) (2026)
- [Image-to-Video Diffusion: From Foundations to Open Frontiers](https://mlsanity.com/paper/2605.17248/) (2026)
- [Magpie: Real-Time World Renderer for Interactive Games](https://mlsanity.com/paper/2608.27168/) (2026)
- [Audio-Visual Intelligence in Large Foundation Models](https://mlsanity.com/paper/2605.04045/) (2026)
- [EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing](https://mlsanity.com/paper/2608.21424/) (2026)
- [Motif-Video 2B: Technical Report](https://mlsanity.com/paper/2604.16503/) (2026)
- [AniMatrix: An Anime Video Generation Model that Thinks in Art, Not Physics](https://mlsanity.com/paper/2605.03652/) (2026)
- [Vorch-Omni: Multi-Task Orchestration of Sight and Sound](https://mlsanity.com/paper/2608.05803/) (2026)
- [InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter](https://mlsanity.com/paper/2608.20910/) (2026)
- [Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction](https://mlsanity.com/paper/2609.23796/) (2026)
- [All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation](https://mlsanity.com/paper/2609.27901/) (2026)

## Abstract (arXiv)

SkyReels V4 is a unified multi modal video foundation model for joint video audio generation, inpainting, and editing. The model adopts a dual stream Multimodal Diffusion Transformer (MMDiT) architecture, where one branch synthesizes video and the other generates temporally aligned audio, while sharing a powerful text encoder based on the Multimodal Large Language Models (MLLM). SkyReels V4 accepts rich multi modal instructions, including text, images, video clips, masks, and audio references. By combining the MLLMs multi modal instruction following capability with in context learning in the video branch MMDiT, the model can inject fine grained visual guidance under complex conditioning, while the audio branch MMDiT simultaneously leverages audio references to guide sound generation. On the video side, we adopt a channel concatenation formulation that unifies a wide range of inpainting style tasks, such as image to video, video extension, and video editing under a single interface, and naturally extends to vision referenced inpainting and editing via multi modal prompts. SkyReels V4 supports up to 1080p resolution, 32 FPS, and 15 second duration, enabling high fidelity, multi shot, cinema level video generation with synchronized audio. To make such high resolution, long-duration generation computationally feasible, we introduce an efficiency strategy: Joint generation of low resolution full sequences and high-resolution keyframes, followed by dedicated super-resolution and frame interpolation models. To our knowledge, SkyReels V4 is the first video foundation model that simultaneously supports multi-modal input, joint video audio generation, and a unified treatment of generation, inpainting, and editing, while maintaining strong efficiency and quality at cinematic resolutions and durations.