mlsanityData to 14 Oct 2026

MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model

arXiv 2606.17800 ↗First posted 16 Jun 2026cs.CVcited by 6

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

As an increasing majority of global video content is consumed on social platforms for interactive social purposes, video generation models built for social worlds are important but largely overlooked by previous studies. In this work, we define the position of social world models and build a prototype model as the first step towards this goal. While previous world models successfully simulate physical environments or gaming world exploration, they remain fundamentally detached from human-centric social dynamics. To bridge this gap as the first step to social world models, we present MaineCoon, the first real-time audio-visual autoregressive model that has 22B parameters and is capable of real-time streaming generation and sub-second interaction, with a record-breaking frame rate of up to 47.5 FPS, on a single GPU. To the best of our knowledge, MaineCoon is also the first real-time audio-visual generation model specifically optimized for social-interactive applications. To enable efficient and stable training, we introduce several novel techniques into MaineCoon, including self-resampling, cross-modal representation alignment, domain-aware preference optimization, and reinforced online-policy distillation (ROPD). We also design the first agentic streaming inference framework that supports thousand-second-scale or even longer generation while mitigating drift with agentic cache management and prompt planing. These innovations significantly accelerate training while optimizing real-time inference performance. We believe this work not only sets a new state-of-the-art (SOTA) performance benchmark for high-quality, low-latency, and long-horizon audio-visual autoregressive models, but also points out the paradigm shift desired for next-generation AI-native social platforms.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 37 of its 69 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

37 of 69 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer14 May
  2. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar

Other references · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2025Qwen-Image Technical Reportcited by 1,044
  3. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  4. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  5. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  6. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  7. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
  8. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  9. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 349
  10. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  11. 2025LongLive: Real-time Interactive Long Video Generationcited by 265
  12. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
21 more
  1. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
  2. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  3. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
  4. 2026A Survey of On-Policy Distillation for Large Language Modelscited by 163
  5. 2025HunyuanVideo 1.5 Technical Reportcited by 160
  6. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixescited by 145
  7. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 115
  8. 2019Music Source Separation in the Waveform Domaincited by 99
  9. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  10. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
  11. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
  12. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  13. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 52
  14. 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 43
  15. 2026DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Modelscited by 40
  16. 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generationcited by 25
  17. 2025MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesiscited by 19
  18. 2026PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformerscited by 17
  19. 2026OmniForcing: Unleashing Real-time Joint Audio-Visual Generationcited by 16
  20. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 7
  21. 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Lengthcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026HelixWorld: A Real-time Interactive Audio-Visual World Model
  2. 2026Waypoint-1.5: A Real-Time Video World Model for Consumer Hardware
  3. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model
  4. 2025TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models
  5. 2026HelloWorld: Enabling Socially Interactive Characters in Video World Models
  6. 2026EchoWM: Open and Enterable Omnimodal World Models
  7. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
  8. 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report