mlsanityData to 14 Oct 2026

Aether: Geometric-Aware Unified World Modeling

arXiv 2503.18945 ↗First posted 24 Mar 2025cs.CV cs.AI cs.LGcited by 97

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

The integration of geometric reconstruction and generative modeling remains a critical challenge in developing AI systems capable of human-like spatial reasoning. This paper proposes Aether, a unified framework that enables geometry-aware reasoning in world models by jointly optimizing three core capabilities: (1) 4D dynamic reconstruction, (2) action-conditioned video prediction, and (3) goal-conditioned visual planning. Through task-interleaved feature learning, Aether achieves synergistic knowledge sharing across reconstruction, prediction, and planning objectives. Building upon video generation models, our framework demonstrates zero-shot synthetic-to-real generalization despite never observing real-world data during training. Furthermore, our approach achieves zero-shot generalization in both action following and reconstruction tasks, thanks to its intrinsic geometric modeling. Notably, even without real-world data, its reconstruction performance is comparable with or even better than that of domain-specific models. Additionally, Aether employs camera trajectories as geometry-informed action spaces, enabling effective action-conditioned prediction and visual planning. We hope our work inspires the community to explore new frontiers in physically-reasonable world modeling and its applications.

Key facts. Cited by 97 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 63 of its 87 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

63 of 87 references.

References · 63

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  10. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
38 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  2. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  3. 2018World Modelscited by 1,513
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  7. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  8. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  9. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  10. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  11. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  12. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  13. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  14. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  15. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  16. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  17. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
  18. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  19. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  20. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  21. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  22. 2023RoMa: Robust Dense Feature Matchingcited by 302
  23. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  24. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
  25. 20243D Reconstruction with Spatial Memorycited by 277
  26. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 263
  27. 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
  28. 2024Global Structure-from-Motion Revisitedcited by 218
  29. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
  30. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
  31. 2020Robust Consistent Video Depth Estimationcited by 188
  32. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  33. 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 181
  34. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
  35. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  36. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  37. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 130
  38. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 42; 2026: 55.

97 papers.

Sort:

By area:

  1. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
  2. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  3. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 139
  4. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  5. 2025Controllable Video Generation: A Surveycited by 98
  6. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  7. 2025TesserAct: Learning 4D Embodied World Modelscited by 94
  8. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  9. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 67
  10. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
  11. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 55
  12. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 54
38 more
  1. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
  2. 2025MindJourney: Test-Time Scaling with World Models for Spatial Reasoningcited by 50
  3. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 40
  4. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 40
  5. 2025RoboScape: Physics-informed Embodied World Modelcited by 40
  6. 2025VRAG: Learning World Models for Interactive Video Generationcited by 38
  7. 2025WinT3R: Window-Based Streaming Reconstruction with Camera Token Poolcited by 35
  8. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  9. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstructioncited by 31
  10. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  11. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  12. 2025PlayerOne: Egocentric World Simulatorcited by 20
  13. 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backendcited by 20
  14. 2025iTACO: Interactable Digital Twins of Articulated Objects from Casually Captured RGBD Videoscited by 19
  15. 2025Dexterous World Modelscited by 18
  16. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  17. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  18. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  19. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 14
  20. 2025Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresightcited by 14
  21. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 13
  22. 2026VGGT-World: Transforming VGGT into an Autoregressive Geometry World Modelcited by 13
  23. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 13
  24. 2026VideoWorld 2: Learning Transferable Knowledge from Real-world Videoscited by 12
  25. 2026Grounding World Simulation Models in a Real-World Metropoliscited by 12
  26. 2026Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulationcited by 12
  27. 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 12
  28. 2026Action Images: End-to-End Policy Learning via Multiview Video Generationcited by 12
  29. 2025How Much 3D Do Video Foundation Models Encode?cited by 11
  30. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 11
  31. 2026ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modelingcited by 10
  32. 2025HERO: Hierarchical Extrapolation and Refresh for Efficient World Modelscited by 9
  33. 2026UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Passcited by 9
  34. 2025VGGT-X: When VGGT Meets Dense Novel View Synthesiscited by 8
  35. 2026MeMix: Writing Less, Remembering More for Streaming 3D Reconstructioncited by 8
  36. 2026GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulationcited by 8
  37. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 8
  38. 2025E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Modelscited by 7

50 most cited of 97 All 97 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning
  2. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction
  3. 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning
  4. 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
  5. 2026Future Dynamic 3D Reconstruction: Toward 3D World Modeling with Disentangled Ego-Motion
  6. 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
  7. 20263D Point World Models: Point Completion Enables More Accurate Dynamics Learning
  8. 2026HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation