mlsanityData to 14 Oct 2026

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

arXiv 2603.19312 ↗First posted 13 Mar 2026cs.LG cs.AIcited by 273

On the map: World models → Latent predictive world models

Abstract (arXiv)

Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse. In this work, we introduce LeWorldModel (LeWM), the first JEPA that trains stably end-to-end from raw pixels using only two loss terms: a next-embedding prediction loss and a regularizer enforcing Gaussian-distributed latent embeddings. This reduces tunable loss hyperparameters from six to one compared to the only existing end-to-end alternative. With ~15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models while remaining competitive across diverse 2D and 3D control tasks. Beyond control, we show that LeWM's latent space encodes meaningful physical structure through probing of physical quantities. Surprise evaluation confirms that the model reliably detects physically implausible events.

Key facts. Cited by 273 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 35 of its 50 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

35 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Temporal Straightening for Latent Planning12 Mar
  2. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
  3. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  4. 2022Temporal Difference Learning for Model Predictive Control9 Mar

Other references · 31

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2016Layer Normalizationcited by 5,669
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2018World Modelscited by 1,513
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  9. 2023Mastering Diverse Domains through World Modelscited by 1,253
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  11. 2020Mastering Atari with Discrete World Modelscited by 1,007
  12. 2018DeepMind Control Suitecited by 956
19 more
  1. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  3. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  4. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  5. 2022Transformers are Sample-Efficient World Modelscited by 328
  6. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  7. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
  8. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  9. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
  10. 2025Training Agents Inside of Scalable World Modelscited by 136
  11. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 94
  12. 2025Intuitive physics understanding emerges from self-supervised pretraining on natural videoscited by 70
  13. 2024Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Maskingcited by 65
  14. 2025AI-Generated Video Detection via Perceptual Straighteningcited by 34
  15. 2024Efficient World Models with Context-Aware Tokenizationcited by 33
  16. 2026EchoJEPA: A Latent Predictive Foundation Model for Echocardiographycited by 13
  17. 2025OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generationcited by 11
  18. 2025stable-pretraining-v1: Foundation Model Research Made Simplecited by 10
  19. 2025Dual Perspectives on Non-Contrastive Self-Supervised Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 273.

273 papers.

Sort:

By area:

  1. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  2. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  3. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  4. 2026Hierarchical Planning with Latent World Modelscited by 43
  5. 2026When Does LeJEPA Learn a World Model?cited by 30
  6. 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluationcited by 20
  7. 2026Sensorimotor World Models: Perception for Action via Inverse Dynamicscited by 20
  8. 2026AdaJEPA: An Adaptive Latent World Modelcited by 20
  9. 2026Predictive but Not Plannable: RC-aux for Latent World Modelscited by 19
  10. 2026World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetrycited by 18
  11. 2026WALL-WM: Carving World Action Modeling at the Event Jointscited by 17
  12. 2026Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Modelscited by 16
38 more
  1. 2026Latent Geometry Beyond Search: Amortizing Planning in World Modelscited by 15
  2. 2026Fast LeWorldModelcited by 15
  3. 2026Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decodingcited by 15
  4. 2026Self-Evolving World Models for LLM Agent Planningcited by 13
  5. 2026Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Modelscited by 11
  6. 2026Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Modelscited by 11
  7. 2026TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulationcited by 11
  8. 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibilitycited by 11
  9. 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 10
  10. 2026SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotorscited by 9
  11. 2026On the Identifiability of Controlled World Modelscited by 9
  12. 2026Pelican-Unify 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Actioncited by 8
  13. 2026ChronoMedicalWorld: A Medical World Model for Learning Patient Trajectories from Longitudinal Care Datacited by 8
  14. 2026World Model Control by Trajectory Reachability Metricscited by 8
  15. 2026PRISM: PRior-guided Imagination Sampling in world Modelscited by 8
  16. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planningcited by 8
  17. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Controlcited by 8
  18. 2026INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Modelscited by 8
  19. 2026Is Forward Prediction Enough? Physical State Grounding for JEPA World Modelscited by 8
  20. 2026LpWM: A Case for Sparse Representations in World Modelscited by 8
  21. 2026X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretrainingcited by 7
  22. 2026SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planningcited by 7
  23. 2026Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planningcited by 7
  24. 2026One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policycited by 6
  25. 2026FF-JEPA: Long-Horizon Planning in World Models with Latent Plannerscited by 6
  26. 2026Orca: The World is in Your Mindcited by 6
  27. 2026Overcoming Statistical Bias in Action-Controllable World Modelscited by 6
  28. 2026PhyLatent: Learning Dynamics-Relevant Representations for JEPA World Modelscited by 6
  29. 2026World-Action Models for Robot Learning and Control: A Surveycited by 6
  30. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  31. 2026Nano World Models: A Minimalist Implementation of Future Video Predictioncited by 5
  32. 2026You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differencescited by 5
  33. 2026A Generalization Theory for JEPA-Based World Modelscited by 5
  34. 2026UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Modelscited by 5
  35. 2026ACID: Action Consistency via Inverse Dynamics for Planning with World Modelscited by 5
  36. 2026QQWorld: Quantile-Quantile Matching for World Model Regularizationcited by 5
  37. 2026Latent World Models with Monotone Planning Costs for Image-Goal Navigationcited by 5
  38. 2026No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Modelscited by 5

50 most cited of 273 All 273 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Joint Embedding Predictive Architectures Focus on Slow Features
  2. 2026A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
  3. 2024Learning and Leveraging World Models in Visual Representation Learning
  4. 2026UWM-JEPA: Predictive World Models That Imagine in Belief Space
  5. 2026VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models
  6. 2026DLLM-JEPA: Joint Embedding Predictive Architectures for Masked Diffusion Language Models
  7. 2025CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Images
  8. 2025seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models