# Temporal Straightening for Latent Planning

arXiv: https://arxiv.org/abs/2603.12231 · HTML: https://mlsanity.com/paper/2603.12231/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 62 of its 76 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 30 of 62)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Time-Contrastive Networks: Self-Supervised Learning from Video](https://mlsanity.com/paper/1704.06888/) (2017)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [Transformers are Sample-Efficient World Models](https://mlsanity.com/paper/2209.00588/) (2022)
- [Deep Visual Foresight for Planning Robot Motion](https://mlsanity.com/paper/1610.00696/) (2016)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)

## Built on by (most cited first, 30 of 38)

- [LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels](https://mlsanity.com/paper/2603.19312/) (2026)
- [LLM-as-a-Verifier: A General-Purpose Verification Framework](https://mlsanity.com/paper/2607.05391/) (2026)
- [AdaJEPA: An Adaptive Latent World Model](https://mlsanity.com/paper/2606.32026/) (2026)
- [Predictive but Not Plannable: RC-aux for Latent World Models](https://mlsanity.com/paper/2605.07278/) (2026)
- [Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Models](https://mlsanity.com/paper/2605.31111/) (2026)
- [SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors](https://mlsanity.com/paper/2606.23444/) (2026)
- [Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations](https://mlsanity.com/paper/2605.14937/) (2026)
- [Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC](https://mlsanity.com/paper/2606.15594/) (2026)
- [ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow](https://mlsanity.com/paper/2607.27924/) (2026)
- [JEDI: Joint Embedding Diffusion World Model for Online Model-Based Reinforcement Learning](https://mlsanity.com/paper/2605.13013/) (2026)
- [Latent Goal Prediction from Language for Model-Based Planning](https://mlsanity.com/paper/2606.20627/) (2026)
- [Music-JEPA: Learning a World Model of Sound from Action](https://mlsanity.com/paper/2607.22000/) (2026)
- [Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning](https://mlsanity.com/paper/2609.03565/) (2026)
- [Grounded World Model for Semantically Generalizable Planning](https://mlsanity.com/paper/2604.11751/) (2026)
- [Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logic](https://mlsanity.com/paper/2605.12651/) (2026)
- [Representation Without Reward: A JEPA Audit for LLM Fine-Tuning](https://mlsanity.com/paper/2605.15394/) (2026)
- [Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models](https://mlsanity.com/paper/2609.21740/) (2026)
- [On Training in Imagination](https://mlsanity.com/paper/2605.06732/) (2026)
- [STEP: Learning STructured Embeddings for Progressive Time Series](https://mlsanity.com/paper/2605.31061/) (2026)
- [Learning Object Manipulation from Scratch via Contrastive Interaction](https://mlsanity.com/paper/2606.11525/) (2026)
- [Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision](https://mlsanity.com/paper/2606.19835/) (2026)
- [Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models](https://mlsanity.com/paper/2607.05390/) (2026)
- [What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models](https://mlsanity.com/paper/2608.14819/) (2026)
- [ARC-Bench: Closed-Loop Replanning Masks Broken Action Ranking in Frozen JEPA World Models](https://mlsanity.com/paper/2609.05461/) (2026)
- [DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation](https://mlsanity.com/paper/2609.10506/) (2026)
- [LePlanner: An Iterative Amortized Controller For World Models](https://mlsanity.com/paper/2609.13845/) (2026)
- [WLA3: World Latent Action Modeling for Semantics, Dynamics, and Kinematics](https://mlsanity.com/paper/2609.15870/) (2026)
- [ViBR-WM: Visual Bayesian Regression for World Modeling](https://mlsanity.com/paper/2609.33844/) (2026)
- [Behavioral Monitoring of JEPA World Models with Jacobian Centroids](https://mlsanity.com/paper/2609.33940/) (2026)
- [AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.34085/) (2026)

## Abstract (arXiv)

Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual features, they are not tailored to planning and contain information irrelevant -- or even detrimental -- to planning. Inspired by the perceptual straightening hypothesis in human visual processing, we introduce temporal straightening to improve representation learning for latent planning. Using a curvature regularizer that encourages locally straightened latent trajectories, we jointly learn an encoder and a predictor of a Joint-Embedding Predictive Architecture (JEPA) world model. We show that reducing curvature this way makes the Euclidean distance in latent space a better proxy for the geodesic distance and improves the conditioning of the planning objective. We demonstrate empirically that temporal straightening makes gradient-based planning more stable and yields significantly higher success rates across a suite of goal-reaching tasks. Our code is available at https://agenticlearning.ai/temporal-straightening.