mlsanityData to 7 Oct 2026

JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

arXiv 2610.00722 ↗First posted 30 Sep 2026cs.LGcited by 0

On the map: World models → Latent predictive world models

Abstract (arXiv)

World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during training. We present JEPA-TTT, which adapts the latent dynamics predictor of a pretrained action-conditioned Joint-Embedding Predictive Architecture world model throughout test time. Self-supervised updates accumulate across episodes, while the visual encoder and reward head remain fixed, preserving the pretrained representation and task objective. Planning requires neither a goal image nor online environment reward. JEPA-TTT uses dense replay, which forms prediction windows at every temporal offset, retains them in a growing buffer, and samples minibatches from that buffer for predictor updates. Across eight dynamics shifts in four continuous-control environments, JEPA-TTT improves planning on every shift. After 500 test-time episodes, it reduces autoregressive latent prediction error by 83% on average and improves planning performance by 153% over the frozen JEPA world model. These results show that persistent self-supervised test-time training can adapt a pretrained latent world model under changed dynamics.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 38 of its 45 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

38 of 45 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026AdaJEPA: An Adaptive Latent World Model30 Jun
  2. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
  3. 2026Temporal Straightening for Latent Planning12 Mar
  4. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
  5. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb
  6. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  7. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2018World Modelscited by 1,444
  5. 2018DeepMind Control Suitecited by 935
  6. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  7. 2022Continual Test-Time Domain Adaptationcited by 520
  8. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  9. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  10. 2021Implicit Behavioral Cloningcited by 442
  11. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  12. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
19 more
  1. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  2. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  3. 2024Navigation World Modelscited by 262
  4. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 227
  5. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  6. 2020Revisiting Fundamentals of Experience Replaycited by 171
  7. 2023Video Language Planningcited by 160
  8. 2025Training Agents Inside of Scalable World Modelscited by 122
  9. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  10. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 93
  11. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  12. 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 82
  13. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  14. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  15. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  16. 2023Finetuning Offline World Models in the Real Worldcited by 39
  17. 2023Test-Time Training on Video Streamscited by 37
  18. 2026WAM-TTT: Steering World-Action Models by Watching Human Play at Test Timecited by 6
  19. 20263D-Belief: Embodied Belief Inference via Generative 3D World Modelingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
  2. 2026LRC-JEPA: Disentangling Dynamics and Residual Context for Efficient World Models
  3. 2026H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
  4. 2026FlexiWorld: Learning and Planning via Flexible Action Chunks Across Multiple Time Scales
  5. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
  6. 2026Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Model
  7. 2026Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning
  8. 2026The Planning Limits of Latent World Models