EchoWM: Open and Enterable Omnimodal World Models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
We present EchoWM, an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech. We organize interaction around camera intent: in first-person scenes, it specifies observer motion, while in third-person scenes, camera--character dynamics are learned from data without view-specific controllers. Discrete commands and continuous poses are mapped to a shared metric-scale relative 6-DoF trajectory, with dataset-level calibration preserving motion magnitude across heterogeneous data. To jointly learn audio-visual generation and trajectory control, we construct a complementary data engine and adopt progressive training followed by autoregressive post-training for long-horizon generation. Extensive evaluations show that model achieves strong trajectory following and high visual quality on public world-model benchmarks, supporting both first- and third-person interaction across varied subjects, and maintaining synchronized environmental sound and speech over long-horizon generation.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 49 of its 52 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
49 of 52 references.
Closest ancestors · 17
Same direction, newest first
- 2026Alaya-EVOKE: From Linear-Scaling Supervision to Endless World13 Aug
- 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU21 Jul
- 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun
- 2026Cosmos 3: Omnimodal World Models for Physical AI1 Jun
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer14 May
- 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
- 2026Solaris: Building a Multiplayer Video World Model in Minecraft25 Feb
- 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory2 Feb
- 2026Advancing Open-source World Models28 Jan
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025Unified Camera Positional Encoding for Controlled Video Generation8 Dec
5 more
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Yume: An Interactive World Generation Model23 Jul
- 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2024Genie: Generative Interactive Environments23 Feb
Other references · 32
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018World Modelscited by 1,513
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
- 2025Qwen3-Omni Technical Reportcited by 497
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
20 more
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
- 2025LongLive: Real-time Interactive Long Video Generationcited by 265
- 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
- 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 195
- 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
- 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 166
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 163
- 2025HunyuanVideo 1.5 Technical Reportcited by 160
- 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
- 2023Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Modelscited by 134
- 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
- 2024FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Soundscited by 110
- 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 98
- 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
- 2024SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Textcited by 26
- 2026Self Gradient Forcing: Native Long Video Extrapolationcited by 17
- 2026OmniForcing: Unleashing Real-time Joint Audio-Visual Generationcited by 16
- 2026Video = World + Event Streamcited by 2
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memorycited by 1
- 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
- 2026Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generationcited by 0
- 2026HappyWorld-Benchcited by 0
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Modelscited by 0
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizoncited by 0
- 2026One-Step Next-Latent Prediction Is Not a World Modelcited by 0
- 2026HelixWorld: A Real-time Interactive Audio-Visual World Modelcited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
- 2026Least Squares for Time Series Forecastingcited by 0
- 2026WorldSonus: Bringing Sound to Worldscited by 0
- 2026SGF+: Decoupling Gradient Flows for Autoregressive Video Generationcited by 0
1 more
- 2026Generative World Simulation with DAO and JINGcited by 0
- 2026Generative World Simulation with DAO and JING14 Oct 2026
- 2026SGF+: Decoupling Gradient Flows for Autoregressive Video Generation7 Oct 2026
- 2026WorldSonus: Bringing Sound to Worlds6 Oct 2026
- 2026Least Squares for Time Series Forecasting1 Oct 2026
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026HelixWorld: A Real-time Interactive Audio-Visual World Model29 Sep 2026
- 2026One-Step Next-Latent Prediction Is Not a World Model28 Sep 2026
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon28 Sep 2026
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models28 Sep 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory21 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memorycited by 1
- 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
- 2026Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generationcited by 0
- 2026HappyWorld-Benchcited by 0
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Modelscited by 0
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizoncited by 0
- 2026HelixWorld: A Real-time Interactive Audio-Visual World Modelcited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
- 2026WorldSonus: Bringing Sound to Worldscited by 0
- 2026WorldSonus: Bringing Sound to Worlds6 Oct 2026
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026HelixWorld: A Real-time Interactive Audio-Visual World Model29 Sep 2026
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon28 Sep 2026
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models28 Sep 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory21 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation3 Sep 2026
- 2026One-Step Next-Latent Prediction Is Not a World Modelcited by 0
- 2026Least Squares for Time Series Forecastingcited by 0
- 2026SGF+: Decoupling Gradient Flows for Autoregressive Video Generationcited by 0
- 2026Generative World Simulation with DAO and JINGcited by 0
- 2026Generative World Simulation with DAO and JING14 Oct 2026
- 2026SGF+: Decoupling Gradient Flows for Autoregressive Video Generation7 Oct 2026
- 2026Least Squares for Time Series Forecasting1 Oct 2026
- 2026One-Step Next-Latent Prediction Is Not a World Model28 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
- 2025Audio-Visual World Models: Learning Physically Grounded Multisensory Dynamics
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model
- 2025OmniCam: Unified Multimodal Video Generation via Camera Control
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound
- 2026Archon: A Unified Multimodal Model for Holistic Digital Human Generation
- 2026Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models