mlsanityData to 7 Oct 2026

EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models

arXiv 2606.12979 ↗First posted 11 Jun 2026cs.LGcited by 0

On the map: World models → Latent predictive world models

Abstract (arXiv)

JEPA-family world models use a static predictor whose weights do not adapt when test-time dynamics diverge from training. We compare two mechanisms for incorporating accumulated experience into a JEPA predictor under distribution shift: operand-side injection, where a compressed experience representation is added as a residual to the predictor's hidden state (EI-JEPA), and operator-side modulation, where the same representation generates low-rank weight deltas via LoRA applied to the predictor's weights (EPM-JEPA). On a pre-registered comparison (Moving MNIST, gravity shift), EPM-JEPA (D_shift^{n=50} = 0.7848 +/- 0.0078, three seeds) differs from EI-JEPA (0.8238) by delta = 4.74% - Outcome C: a null result - by our stated criterion, a valid outcome. As a secondary, non-pre-registered observation, EPM-JEPA improves 1.90% over a no-memory baseline (0.8000), consistently across seeds, while EI-JEPA underperforms the baseline, indicating the benefit is specific to weight-level modulation. Our primary contribution is a mechanism analysis: the D_shift^{n=50} trajectory reflects three independent dynamical processes - buffer cycling, EMA target drift, and an intrinsic LoRA settling transient of +0.021 - rather than convergence to equilibrium. These findings motivate PEM-JEPA, a physics-grounded successor addressing this dynamical-peak limitation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 12 of its 17 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

12 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan

Other references · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  6. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  7. 2016HyperNetworkscited by 1,003
  8. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  9. 2014Neural Turing Machinescited by 630
  10. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  11. 2022Memorizing Transformerscited by 219

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Branch-JEPA: Finite-Support Predictive Distributions for JEPA World Models
  2. 2026JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics
  3. 2026JEPA-Anything: Learning Predictive Models across Different Worlds
  4. 2026A Generalization Theory for JEPA-Based World Models
  5. 2026Do-JEPA: From Masking to Intervention in Latent World Models
  6. 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency
  7. 2026UWM-JEPA: Predictive World Models That Imagine in Belief Space
  8. 2026Orthogonal JEPA: Factorized Predictive States for Latent World Models