mlsanityData to 7 Oct 2026

Temporal Straightening for Latent Planning

arXiv 2603.12231 ↗First posted 12 Mar 2026cs.LGcited by 32

On the map: World models → Latent predictive world models

Abstract (arXiv)

Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual features, they are not tailored to planning and contain information irrelevant -- or even detrimental -- to planning. Inspired by the perceptual straightening hypothesis in human visual processing, we introduce temporal straightening to improve representation learning for latent planning. Using a curvature regularizer that encourages locally straightened latent trajectories, we jointly learn an encoder and a predictor of a Joint-Embedding Predictive Architecture (JEPA) world model. We show that reducing curvature this way makes the Euclidean distance in latent space a better proxy for the geodesic distance and improves the conditioning of the planning objective. We demonstrate empirically that temporal straightening makes gradient-based planning more stable and yields significantly higher success rates across a suite of goal-reaching tasks. Our code is available at https://agenticlearning.ai/temporal-straightening.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 62 of its 76 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

62 of 76 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?30 Dec
  2. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
  3. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb
  4. 2022Temporal Difference Learning for Model Predictive Control9 Mar
  5. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  12. 2018World Modelscited by 1,444
38 more
  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  2. 2025DINOv3cited by 1,297
  3. 2023Mastering Diverse Domains through World Modelscited by 1,193
  4. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  5. 2020Mastering Atari with Discrete World Modelscited by 990
  6. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  7. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  8. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  9. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  10. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  12. 2022Transformers are Sample-Efficient World Modelscited by 317
  13. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  14. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  15. 2024Navigation World Modelscited by 262
  16. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 227
  17. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  18. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  19. 2017Learning Koopman Invariant Subspaces for Dynamic Mode Decompositioncited by 156
  20. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  21. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  22. 2017Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systemscited by 125
  23. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  24. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  25. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  26. 2018Learning Plannable Representations with Causal InfoGANcited by 56
  27. 2019Learning Robotic Manipulation through Visual Planning and Actingcited by 39
  28. 2017Robust Locally-Linear Controllable Embeddingcited by 37
  29. 2025AI-Generated Video Detection via Perceptual Straighteningcited by 34
  30. 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 33
  31. 2023Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural languagecited by 32
  32. 2025Whole-Body Conditioned Egocentric Video Predictioncited by 30
  33. 2015Learning to Linearize Under Uncertaintycited by 25
  34. 2020Plan2Vec: Unsupervised Representation Learning by Latent Planscited by 21
  35. 2019Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Controlcited by 17
  36. 2020Predictive Coding for Locally-Linear Controlcited by 17
  37. 2022On the Learning and Learnability of Quasimetricscited by 15
  38. 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 15

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 32.

32 papers.

Sort:

By area:

  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  2. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 21
  3. 2026Predictive but Not Plannable: RC-aux for Latent World Modelscited by 16
  4. 2026AdaJEPA: An Adaptive Latent World Modelcited by 16
  5. 2026Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Modelscited by 11
  6. 2026SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotorscited by 9
  7. 2026ODEWorld: A Continuous Predictive Architecture via Physical-Time Flowcited by 4
  8. 2026Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representationscited by 3
  9. 2026JEDI: Joint Embedding Diffusion World Model for Online Model-Based Reinforcement Learningcited by 2
  10. 2026Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPCcited by 2
  11. 2026Grounded World Model for Semantically Generalizable Planningcited by 1
  12. 2026Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logiccited by 1
20 more
  1. 2026Representation Without Reward: A JEPA Audit for LLM Fine-Tuningcited by 1
  2. 2026Latent Goal Prediction from Language for Model-Based Planningcited by 1
  3. 2026Music-JEPA: Learning a World Model of Sound from Actioncited by 1
  4. 2026Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planningcited by 1
  5. 2026On Training in Imaginationcited by 0
  6. 2026STEP: Learning STructured Embeddings for Progressive Time Seriescited by 0
  7. 2026Neural Events: Discrete Asynchronous Autoencoders for Event-Based Visioncited by 0
  8. 2026Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Modelscited by 0
  9. 2026What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Modelscited by 0
  10. 2026ARC-Bench: Closed-Loop Replanning Masks Broken Action Ranking in Frozen JEPA World Modelscited by 0
  11. 2026DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulationcited by 0
  12. 2026LePlanner: An Iterative Amortized Controller For World Modelscited by 0
  13. 2026WLA3: World Latent Action Modeling for Semantics, Dynamics, and Kinematicscited by 0
  14. 2026Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Modelscited by 0
  15. 2026ViBR-WM: Visual Bayesian Regression for World Modelingcited by 0
  16. 2026Behavioral Monitoring of JEPA World Models with Jacobian Centroidscited by 0
  17. 2026AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Drivingcited by 0
  18. 2026Learning to Plan from Random Explorationcited by 0
  19. 2026JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shiftscited by 0
  20. 2026Learning Commute-Time-Preserving World Models for Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Control-Geometry Straightening for Sampling-Based Latent Planning
  2. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
  3. 2026Beyond a single latent space: a dual-latent world model for long-horizon planning
  4. 2026ATLAS: Aligned Transport of Latent Structure for Reliable World Model Planning
  5. 2026Anisotropic Representations Improve Planning in JEPA World Models
  6. 2026How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
  7. 2026Learning Invariant Visual Representations for Planning with Joint-Embedding Predictive World Models
  8. 2021Temporal Predictive Coding For Model-Based Planning In Latent Space