mlsanityData to 7 Oct 2026

Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency

arXiv 2608.12939 ↗First posted 13 Aug 2026cs.LGcited by 2

On the map: World models → Latent predictive world models

Abstract (arXiv)

Joint-embedding predictive architectures (JEPAs) learn world models that predict in a compact latent space rather than in pixels, reducing the pressure to model nuisance appearance. Yet this provides no guarantee against visual perturbations: they can still alter the encoded representation and affect subsequent action-conditioned predictions. Bisimulation captures this requirement precisely: two observations should be treated as the same state only when their action-conditioned consequences agree. Guided by this criterion, we introduce Action-Conditioned Predictive Consistency (ACPC), a diagnostic that measures how far a clean history and a visually perturbed view of it diverge after being rolled forward under the same action sequence. We prove that this divergence bounds the perturbation-induced change in multi-step prediction error and planner cost. Building on pairwise ACPC, we define two complementary measures: the Invariance Radius (IR) summarizes clean-perturbed rollout spread, while the Separation Rate (SR) checks whether different states remain distinguishable after rollout. Experiments on four visual control tasks show that pairwise ACPC predicts perturbation-induced prediction and cost changes. On LeWM, the IR-SR screen transfers across tasks, and the joint diagnostic remains informative under blur and resize. PLDM exhibits similar diagnostic trends under a different architecture.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 51 of its 60 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

51 of 60 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026A Control Theory of Predictability in Latent World Models11 Jul
  2. 2026Imagined Rollouts are Kinematic, Not Dynamic: A Diagnosis of Long-Horizon World-Model Failure7 Jul
  3. 2026ACID: Action Consistency via Inverse Dynamics for Planning with World Models2 Jul
  4. 2026Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding30 Jun
  5. 2026ATM: Action-Consistency Transfer Matrix for Diagnosing and Improving Latent World Models8 Jun
  6. 2026When Does LeJEPA Learn a World Model?25 May
  7. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
  8. 2026Learning Invariant Visual Representations for Planning with Joint-Embedding Predictive World Models20 Feb
  9. 2025seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models6 May
  10. 2024Revisiting Feature Prediction for Learning Visual Representations from Video15 Feb
  11. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  12. 2022Joint Embedding Predictive Architectures Focus on Slow Features20 Nov

Other references · 39

  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  2. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  3. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  4. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  5. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  6. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  7. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  8. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  9. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 101
  10. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
  11. 2020Objective Mismatch in Model-based Reinforcement Learningcited by 87
  12. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
27 more
  1. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
  2. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
  3. 2021Learning Task Informed Abstractionscited by 58
  4. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 54
  5. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  6. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  7. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  8. 2022Understanding Self-Predictive Learning for Reinforcement Learningcited by 40
  9. 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 33
  10. 2024How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networkscited by 29
  11. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 20
  12. 2024AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractorscited by 10
  13. 2026Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Modelscited by 10
  14. 2023A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learningcited by 9
  15. 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 9
  16. 2026What Makes Video World Model Latents Action-Relevant: Prediction over Reconstructioncited by 8
  17. 2024A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learningcited by 7
  18. 2026MWM: Mobile World Models for Action-Conditioned Consistent Predictioncited by 7
  19. 2024Bisimulation metric for Model Predictive Controlcited by 6
  20. 2024Policy-shaped prediction: avoiding distractions in model-based reinforcement learningcited by 6
  21. 2025Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Controlcited by 6
  22. 2026WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agentscited by 6
  23. 2026World Models as Group Actionscited by 3
  24. 2025Calibrated Value-Aware Model Learning with Probabilistic Environment Modelscited by 2
  25. 2026ARB4WM: An Adversarial Robustness Benchmark for World Models in Continuous Controlcited by 2
  26. 2026Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world modelscited by 2
  27. 2023VIBR: Learning View-Invariant Value Functions for Robust Visual Controlcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Beyond Task Success: Stage-Wise Reliability of World Model Planning under Sensing Degradationcited by 0
  2. 2026Beyond a single latent space: a dual-latent world model for long-horizon planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State
  2. 2026UniJEPA: A Unified Joint-Embedding Predictive Architecture for Task-Agnostic Visual World Modeling
  3. 2026No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Models
  4. 2026Flow-JEPA: Flow Matching for Robust Latent Dynamics in JEPA World Models
  5. 2026AVL-JEPA: Preventing Causal Dynamics Information Collapse In Joint Embedding Predictive Architecture World Models
  6. 2026UWM-JEPA: Predictive World Models That Imagine in Belief Space
  7. 2026V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
  8. 2026JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics