mlsanityData to 7 Oct 2026

V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning

arXiv 2506.09985 ↗First posted 11 Jun 2025cs.AI cs.CV cs.LGcited by 738

On the map: World models → Embodied and robot world models

Abstract (arXiv)

A major challenge for modern AI is to learn to understand the world and learn to act largely by observation. This paper explores a self-supervised approach that combines internet-scale video data with a small amount of interaction data (robot trajectories), to develop models capable of understanding, predicting, and planning in the physical world. We first pre-train an action-free joint-embedding-predictive architecture, V-JEPA 2, on a video and image dataset comprising over 1 million hours of internet video. V-JEPA 2 achieves strong performance on motion understanding (77.3 top-1 accuracy on Something-Something v2) and state-of-the-art performance on human action anticipation (39.7 recall-at-5 on Epic-Kitchens-100) surpassing previous task-specific models. Additionally, after aligning V-JEPA 2 with a large language model, we demonstrate state-of-the-art performance on multiple video question-answering tasks at the 8 billion parameter scale (e.g., 84.0 on PerceptionTest, 76.9 on TempCompass). Finally, we show how self-supervised learning can be applied to robotic planning tasks by post-training a latent action-conditioned world model, V-JEPA 2-AC, using less than 62 hours of unlabeled robot videos from the Droid dataset. We deploy V-JEPA 2-AC zero-shot on Franka arms in two different labs and enable picking and placing of objects using planning with image goals. Notably, this is achieved without collecting any data from the robots in these environments, and without any task-specific training or reward. This work demonstrates how self-supervised learning from web-scale data and a small amount of robot interaction data can yield a world model capable of planning in the physical world.

Key facts. Cited by 738 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 102 of its 129 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

102 of 129 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025FLARE: Robot Learning with Implicit World Modeling21 May
  2. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
  3. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation25 Sep
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  5. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physics23 Jun

Other references · 97

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  12. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
38 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  8. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  9. 2018World Modelscited by 1,444
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  11. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  12. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  13. 2023Mastering Diverse Domains through World Modelscited by 1,193
  14. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  15. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  16. 2017Hindsight Experience Replaycited by 1,096
  17. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  18. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  19. 2021Scaling Vision Transformerscited by 993
  20. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  21. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  22. 2023Vision Transformers Need Registerscited by 844
  23. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  24. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  25. 2024Genie: Generative Interactive Environmentscited by 706
  26. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  27. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  28. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  29. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  30. 2020Rescaling Egocentric Visioncited by 529
  31. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  32. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  33. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  34. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  35. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  36. 2023Learning Interactive Real-World Simulatorscited by 440
  37. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  38. 2022Temporal Difference Learning for Model Predictive Controlcited by 375

50 most cited of 97 All 97 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 113; 2026: 625.

738 papers.

Sort:

By area:

  1. 2026World Action Models are Zero-shot Policiescited by 411
  2. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  3. 2025Video models are zero-shot learners and reasonerscited by 208
  4. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  6. 2026Advancing Open-source World Modelscited by 144
  7. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  8. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  9. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
  10. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  11. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  12. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
38 more
  1. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
  2. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  3. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 86
  4. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  5. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 68
  6. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  7. 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 66
  8. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 65
  9. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  10. 20253D and 4D World Modeling: A Surveycited by 64
  11. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
  12. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 60
  13. 2025World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Trainingcited by 57
  14. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  15. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  16. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  17. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  18. 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policiescited by 46
  19. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 43
  20. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving Worldcited by 43
  21. 2026Learning Latent Action World Models In The Wildcited by 42
  22. 2026World Model for Robot Learning: A Comprehensive Surveycited by 42
  23. 2025Scaling Agent Learning via Experience Synthesiscited by 40
  24. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  25. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Reviewcited by 38
  26. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
  27. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  28. 2025World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulationcited by 35
  29. 2026Do World Action Models Generalize Better than VLAs? A Robustness Studycited by 35
  30. 2026Hierarchical Planning with Latent World Modelscited by 34
  31. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 32
  32. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  33. 2026Temporal Straightening for Latent Planningcited by 32
  34. 2025Whole-Body Conditioned Egocentric Video Predictioncited by 30
  35. 2026Interactive World Simulator for Robot Policy Training and Evaluationcited by 27
  36. 2026When Does LeJEPA Learn a World Model?cited by 27
  37. 2025NORA-1.5: A Vision-Language-Action Model Trained using World Model- and Action-based Preference Rewardscited by 26
  38. 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 26

50 most cited of 738 All 738 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
  2. 2026A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
  3. 2026JEPA-WAM: Connecting Generated Visual Instructions to World Action Models through JEPA Latent Representations
  4. 2025Large Video Planner Enables Generalizable Robot Control
  5. 2024Latent Action Pretraining from Videos
  6. 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI
  7. 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data
  8. 2025Latent Action Pretraining Through World Modeling