# LpWM: A Case for Sparse Representations in World Models

arXiv: https://arxiv.org/abs/2608.22764 · HTML: https://mlsanity.com/paper/2608.22764/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 24 Aug 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 29 of 29)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Whitening for Self-Supervised Representation Learning](https://mlsanity.com/paper/2007.06346/) (2020)
- [LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels](https://mlsanity.com/paper/2603.19312/) (2026)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control](https://mlsanity.com/paper/1611.03537/) (2016)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [A Spectral Condition for Feature Learning](https://mlsanity.com/paper/2310.17813/) (2023)
- [Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models](https://mlsanity.com/paper/2502.14819/) (2025)
- [Hierarchical Planning with Latent World Models](https://mlsanity.com/paper/2604.03208/) (2026)
- [Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2303.03307/) (2023)
- [stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation](https://mlsanity.com/paper/2605.21800/) (2026)
- [AdaJEPA: An Adaptive Latent World Model](https://mlsanity.com/paper/2606.32026/) (2026)
- [A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2602.03604/) (2026)
- [Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations](https://mlsanity.com/paper/2602.01456/) (2026)
- [Learning Robust Dynamics through Variational Sparse Gating](https://mlsanity.com/paper/2210.11698/) (2022)
- [Improving Pre-trained Self-Supervised Embeddings Through Effective Entropy Maximization](https://mlsanity.com/paper/2411.15931/) (2024)
- [Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization](https://mlsanity.com/paper/2602.14272/) (2026)
- [SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry](https://mlsanity.com/paper/2608.16287/) (2026)
- [On the Existence of Koopman Linear Embeddings for Controlled Nonlinear Systems](https://mlsanity.com/paper/2602.14537/) (2026)
- [Data-Driven Deep Learning Based Feedback Linearization of Systems with Unknown Dynamics](https://mlsanity.com/paper/2303.08926/) (2023)

## Built on by (most cited first, 8 of 8)

- [Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems](https://mlsanity.com/paper/2608.29057/) (2026)
- [Adaptive Latent Capacity for World Models](https://mlsanity.com/paper/2609.32921/) (2026)
- [AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.34085/) (2026)
- [Control-Geometry Straightening for Sampling-Based Latent Planning](https://mlsanity.com/paper/2609.35603/) (2026)
- [Anisotropic Representations Improve Planning in JEPA World Models](https://mlsanity.com/paper/2609.37441/) (2026)
- [When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models](https://mlsanity.com/paper/2610.05550/) (2026)
- [H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning](https://mlsanity.com/paper/2610.06805/) (2026)
- [Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models](https://mlsanity.com/paper/2610.07540/) (2026)

## Abstract (arXiv)

Joint-embedding predictive architectures (JEPAs) learn latent dynamics for planning and avoid representation collapse by matching features to maximum-entropy distributions such as isotropic Gaussians, yielding dense representations. However, it is unclear whether dense representations are the most favorable geometry for modeling dynamics. In this work, we ask whether a different geometry, sparse representations, can make action-conditioned latent dynamics easier to model, and what dynamical structure emerges from such representations. We first show that nonlinear Lipschitz dynamics can be approximated arbitrarily well by action-conditioned linear dynamics in a sufficiently high-dimensional one-hot latent space, with rollout error vanishing as the dimension grows. This motivates distributed sparse representations as a practical relaxation of one-hot sparsity. We introduce LpWorldModel (LpWM), a JEPA model regularized with Rectified Distribution Matching Regularization (RDMReg) to match encoder features to a Rectified Generalized Gaussian distribution, yielding non-negative sparse codes. Empirically, sparsity lowers the predictor complexity required for successful planning: on PushT, sparse LpWM outperforms dense LeWM by up to 57% in planning success at intermediate predictor capacities. This advantage also extends beyond Gaussian distribution matching, with LpWM outperforming dense VICReg representations across multiple predictor families. We further find that the learned sparse representations are mode-factored, with support encoding discrete dynamical regimes and feature magnitudes capturing continuous within-regime state. Together, these results suggest that sparse representations can reduce the predictor complexity required for control while revealing interpretable structure.