mlsanityData to 13 Oct 2026

Joint Embedding Predictive Architectures Focus on Slow Features

arXiv 2211.10831 ↗First posted 20 Nov 2022cs.LGcited by 27

On the map: World models → Latent predictive world models

Abstract (arXiv)

Many common methods for learning a world model for pixel-based environments use generative architectures trained with pixel-level reconstruction objectives. Recently proposed Joint Embedding Predictive Architectures (JEPA) offer a reconstruction-free alternative. In this work, we analyze performance of JEPA trained with VICReg and SimCLR objectives in the fully offline setting without access to rewards, and compare the results to the performance of the generative architecture. We test the methods in a simple environment with a moving dot with various background distractors, and probe learned representations for the dot's location. We find that JEPA methods perform on par or better than reconstruction when distractor noise changes every time step, but fail when the noise is fixed. Furthermore, we provide a theoretical explanation for the poor performance of JEPA-based methods with fixed noise, highlighting an important limitation.

Key facts. Cited by 27 papers in the map (data through 13 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 34 of its 56 references that are in the map. First posted on arXiv 20 Nov 2022.

Builds on

34 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 33

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,712
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,492
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,730
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,860
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,850
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,826
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,620
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,397
  10. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  11. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,269
  12. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,033
21 more
  1. 2020Mastering Atari with Discrete World Modelscited by 996
  2. 2020What Makes for Good Views for Contrastive Learning?cited by 844
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 747
  4. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 455
  5. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 431
  6. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 375
  7. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  8. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 311
  9. 2020Decoupling Representation Learning from Reinforcement Learningcited by 264
  10. 2019Video Representation Learning by Dense Predictive Codingcited by 175
  11. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  12. 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
  13. 2019Unsupervised State Representation Learning in Ataricited by 144
  14. 2020Intriguing Properties of Contrastive Lossescited by 129
  15. 2021Can contrastive learning avoid shortcut solutions?cited by 114
  16. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  17. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  18. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  19. 2020Watching the World Go By: Representation Learning from Unlabeled Videoscited by 59
  20. 2020Representation Learning with Video Deep InfoMaxcited by 17
  21. 2022Light-weight probing of unsupervised representations for Reinforcement Learningcited by 15

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 3; 2026: 18.

27 papers.

Sort:

By area:

  1. 2023A Cookbook of Self-Supervised Learningcited by 249
  2. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 84
  3. 2024How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networkscited by 30
  4. 2026When Does LeJEPA Learn a World Model?cited by 29
  5. 2024Hierarchical World Models as Visual Whole-Body Humanoid Controllerscited by 25
  6. 2023Gradient-based Planning with World Modelscited by 18
  7. 2026A Lightweight Library for Energy-Based Joint-Embedding Predictive Architecturescited by 14
  8. 2026Latent Geometry Beyond Search: Amortizing Planning in World Modelscited by 14
  9. 2026Fast LeWorldModelcited by 14
  10. 2026Learning Invariant Visual Representations for Planning with Joint-Embedding Predictive World Modelscited by 13
  11. 2024T-JEPA: A Joint-Embedding Predictive Architecture for Trajectory Similarity Computationcited by 12
  12. 2026Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Modelscited by 11
15 more
  1. 2024Towards Principled Representation Learning from Videos for Reinforcement Learningcited by 10
  2. 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistencycited by 3
  3. 2025Discrete JEPA: Learning Discrete Token Representations without Reconstructioncited by 2
  4. 2025Learning State-Space Models of Dynamic Systems from Arbitrary Data using Joint Embedding Predictive Architecturescited by 2
  5. 2026LLM2Vec-Gen: Generative Embeddings from Large Language Modelscited by 2
  6. 2026AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modelingcited by 2
  7. 2026JEPA-Anything: Learning Predictive Models across Different Worldscited by 1
  8. 2026Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Modelscited by 0
  9. 2026Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Modelscited by 0
  10. 2026AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Controlcited by 0
  11. 2026Adaptive Latent Capacity for World Modelscited by 0
  12. 2026LRC-JEPA: Disentangling Dynamics and Residual Context for Efficient World Modelscited by 0
  13. 2026Frozen in a Frame: The Velocity Blind Spot in JEPA World Modelscited by 0
  14. 2026EpicWorldModel: Exploration-driven Planning with Latent World Modelscited by 0
  15. 2026H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Denoising with a Joint-Embedding Predictive Architecture
  2. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
  3. 2026UniJEPA: A Unified Joint-Embedding Predictive Architecture for Task-Agnostic Visual World Modeling
  4. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
  5. 2026The JEPA Predictor: A Transferable Operator for Occluded Feature Completion
  6. 2025JEPA for RL: Investigating Joint-Embedding Predictive Architectures for Reinforcement Learning
  7. 2026Mask-Aware Execution for Efficient JEPA Training
  8. 2026Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State