mlsanityData to 7 Oct 2026

Cosmos World Foundation Model Platform for Physical AI

arXiv 2501.03575 ↗First posted 7 Jan 2025cs.CV cs.AI cs.LGcited by 833

On the map: World models → World models for language agents

Abstract (arXiv)

Physical AI needs to be trained digitally first. It needs a digital twin of itself, the policy model, and a digital twin of the world, the world model. In this paper, we present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups. We position a world foundation model as a general-purpose world model that can be fine-tuned into customized world models for downstream applications. Our platform covers a video curation pipeline, pre-trained world foundation models, examples of post-training of pre-trained world foundation models, and video tokenizers. To help Physical AI builders solve the most critical problems of our society, we make Cosmos open-source and our models open-weight with permissive licenses available via https://github.com/nvidia-cosmos/cosmos-predict1.

Key facts. Cited by 833 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 87 of its 252 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

87 of 252 references.

References · 87

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2016Layer Normalizationcited by 5,579
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2024Qwen2.5 Technical Reportcited by 4,713
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  11. 2013Auto-Encoding Variational Bayescited by 3,980
  12. 2023Mistral 7Bcited by 3,065
38 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  8. 2018World Modelscited by 1,444
  9. 2023Mastering Diverse Domains through World Modelscited by 1,193
  10. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  12. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  13. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  14. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  15. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  16. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  17. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  18. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  19. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  20. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  21. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  22. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  23. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  24. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  25. 2023Learning Interactive Real-World Simulatorscited by 440
  26. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  27. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
  28. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  29. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  30. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  31. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  32. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 295
  33. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  34. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  35. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  36. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  37. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  38. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 337; 2026: 496.

833 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  4. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  5. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  6. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
  7. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  8. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
  9. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  10. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  11. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  12. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
38 more
  1. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  2. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  3. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  4. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  5. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  6. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
  7. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
  8. 2025Do generative video models understand physical principles?cited by 151
  9. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  10. 2026Advancing Open-source World Modelscited by 144
  11. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  12. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 141
  13. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  14. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  15. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  16. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  17. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  18. 2025DDT: Decoupled Diffusion Transformercited by 100
  19. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
  20. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  21. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraftcited by 93
  22. 2025Controllable Video Generation: A Surveycited by 93
  23. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  24. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 88
  25. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 87
  26. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  27. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 85
  28. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  29. 2025Seaweed-7B: Cost-Effective Training of Video Generation Foundation Modelcited by 81
  30. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 80
  31. 2025Improving the Diffusability of Autoencoderscited by 74
  32. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  33. 2025Yume: An Interactive World Generation Modelcited by 73
  34. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  35. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 70
  36. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 68
  37. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 68
  38. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68

50 most cited of 833 All 833 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026From World Models to World Action Models: A Concise Tutorial for Robotics
  2. 2026A Tutorial on World Models and Physical AI
  3. 2025Foundation Model Driven Robotics: A Comprehensive Review
  4. 2024Real-World Robot Applications of Foundation Models: A Review
  5. 2023Foundation Models in Robotics: Applications, Challenges, and the Future
  6. 2025Position: Foundation Models Need Digital Twin Representations
  7. 2026WorldAgents: Can Foundation Image Models be Agents for 3D World Models?
  8. 2024Pandora: Towards General World Model with Natural Language Actions and Video States