LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows
On the map: Diffusion models → Video generation
also in 1 more
Abstract (arXiv)
Video diffusion models are stochastic and hard to control: precise content often requires repeated sampling without guaranteed success, and long-horizon scenes drift in appearance, interactions, and temporal coherence. Agentic visual creation provides explicit references, editable 3D scenes, or executable game states for stable control, but does not by itself guarantee high object or character fidelity. Combining the two can enable stable, high-quality generation. To realize this combination, we present LynnReal-Omni, a native multimodal video generation framework built on a 32B shared multimodal diffusion transformer that unifies text-to-video, image-conditioned generation, reference-guided generation, structural control, editing, degraded video restoration, and long-video generation. It accepts heterogeneous visual inputs, including appearance references, editable 3D renders, and game recordings, allowing agents to compose visual conditions within a unified model. We also train a dedicated 27B Flash shared multimodal diffusion transformer for real-time rendering. We build a systematic data pipeline for video cleaning, subject association, multimodal annotation, and aligned control construction, yielding a curated corpus of multi-shot audiovisual segments, and introduce MSAVP, a 100-prompt, 20-metric evaluation design that separates instruction following, generating plausibility, visual quality, temporal behavior, and audio coordination. LynnReal-Omni-Flash further reduces inference cost through model and decoding acceleration, including a lightweight VAE decoder; on one H100, warm generation and decoding of a 22-frame 540p video take 843 ms with LynnReal-Omni and 377 ms with Flash. These results provide a foundation for real-time streaming video generation, making LynnReal-Omni a unified, controllable, and efficient basis for agentic visual creation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 28 of its 55 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
28 of 55 references.
Closest ancestors · 5
Same direction, newest first
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generation2 Oct
- 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer29 Sep
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion9 Jun
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
Other references · 23
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2025Qwen3-Omni Technical Reportcited by 497
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
11 more
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2020PP-OCR: A Practical Ultra Lightweight OCR Systemcited by 151
- 2025Learning Few-Step Diffusion Models by Trajectory Distribution Matchingcited by 57
- 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 49
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 39
- 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 36
- 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 23
- 2026Code World Model: Coding Agent as World Braincited by 14
- 2026Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generationcited by 13
- 2026GenClaw: Code-Driven Agentic Image Generationcited by 11
- 2026MOSS Transcribe Diarize Technical Reportcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing
- 2025CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusion
- 2026UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors
- 2026Lighting-grounded Video Generation with Renderer-based Agent Reasoning
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
- 2026VINO: A Unified Visual Generator with Interleaved OmniModal Context
- 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
- 2024Kubrick: Multimodal Agent Collaborations for Synthetic Video Generation