mlsanityData to 7 Oct 2026

LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC

arXiv 2610.12407 ↗First posted 8 Oct 2026cs.RO cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

World action models (WAMs) predict actions and future observations, typically from a reconstruction-based representation that carries noisy, redundant information which can complicate downstream predictions. We introduce LeWAM, a bidirectional transformer for forward, backward, inverse dynamics and policy prediction, on a decoder-free JEPA latent trained end-to-end through all four modes. We see the following benefits: 1) Alignment: linear probes read robot and object state from LeWAM's latent better than from a regular Le World Model (a forward-only JEPA world model), while the latent ignores visual distractors as well as LeWM does and far better than a reconstruction-based WAM. 2) Acting: Closed-loop evaluations of LeWAM match a regular flow-matching policy trained on the same encoder at matched size, while also providing a world model. 3) Planning: Sampling raw actions when planning with WAMs lets MPC exploit dynamics-model inaccuracies; planning in the noise space of the policy head instead improves the closed-loop performance of these WAMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 42 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

19 of 42 references.

References · 19

  1. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  3. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  4. 2026World Action Models are Zero-shot Policiescited by 411
  5. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  6. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  7. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  8. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  9. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 227
  10. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  11. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  12. 2020Model-Based Offline Planningcited by 126
7 more
  1. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  2. 2024Diffusion Model Predictive Controlcited by 35
  3. 2018Natural Environment Benchmarks for Reinforcement Learningcited by 32
  4. 2025Inference-Time Enhancement of Generative Robot Policies via Predictive World Modelingcited by 22
  5. 2026You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vectorcited by 5
  6. 2026Lagrangian Perturbation Diffusion Steering: Latent Reinforcement Learning for Generative Policiescited by 2
  7. 2026FlowMPC: Improving Flow Matching policies with World Modelscited by 1

Built on by

Not cited yet.