mlsanityData to 14 Oct 2026

Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equations

arXiv 2607.21644 ↗First posted 21 Jul 2026cs.LG cs.SY eess.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a goal-agnostic control framework for partial differential equations (PDEs) built around an end-to-end joint-embedding predictive architecture (JEPA). A lightweight 2D vision-transformer (ViT) and action-conditioned latent dynamics are trained offline without a reward or downstream goal, before being frozen and reused by a model-predictive path integral (MPPI) controller. We minimize a control objective in the latent space, initially expressed via the L2 distance and additionally illustrate the benefit of recasting the control objective in terms of an explicit physical observable when available. By instead minimizing the tracking error for a learned linear kinetic-energy (KE) probe on the frozen latent-state rollouts, we demonstrate the ability to reproduce the control of held-out trajectories with R2=0.989, while requiring no change to the underlying world model. For a controlled 2D Navier--Stokes benchmark, using a KE-probe within MPPI planning improves the mean native reward from -12.08±0.86 for latent-L2 tracking to -10.90±0.91 (95% CI), all while lowering last-quarter velocity-field RMSE from 0.0765 to 0.0692. Across three intentionally withheld, dissimilar, aperiodic targets, KE planning lowers late field RMSE by 53% relative to latent-L2 planning (0.0220 versus 0.0469), winning across 30 paired comparisons. The same frozen model also supports stabilization around a steady-state configuration via direct regulation of KE, achieving 2.7% mean relative error. While the latent probe proves brittle to measurement noise and missing pixels, our findings support the claim that latent dynamics can remain flexible and goal-agnostic, particularly when calibrated observables (granted they guarantee unique continuation) are a suitable objective for state control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

32 of 40 references.

References · 32

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2018World Modelscited by 1,513
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  10. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  11. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  12. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
20 more
  1. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  2. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  3. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  4. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  5. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 273
  6. 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 140
  7. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 140
  8. 2020Learning to Control PDEs with Differentiable Physicscited by 132
  9. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  10. 2018Deep Dynamical Modeling and Control of Unsteady Fluid Flowscited by 68
  11. 2021Optimal control of PDEs using physics-informed neural networkscited by 52
  12. 2021Solving PDE-constrained Control Problems Using Operator Learningcited by 42
  13. 2022Learning to Accelerate Partial Differential Equations via Latent Global Evolutioncited by 39
  14. 2017Koopman operator-based model reduction for switched-system control of PDEscited by 31
  15. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 29
  16. 2023Neural Operators for Bypassing Gain and Control Computations in PDE Backsteppingcited by 29
  17. 2024DiffPhyCon: A Generative Approach to Control Complex Physical Systemscited by 17
  18. 2026Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decodingcited by 15
  19. 2024PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equationscited by 10
  20. 2026SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotorscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State
  2. 2026LeWAM: End-to-End World and Action Modeling with JEPAs
  3. 2026JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics
  4. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
  5. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
  6. 2026LeHami: Symplectic Latent Dynamics for World Models
  7. 2026Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC
  8. 2026Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models