mlsanityData to 14 Oct 2026

Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC

arXiv 2606.15594 ↗First posted 14 Jun 2026cs.RO cs.AI cs.CVcited by 4

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

We present SLS^2, a framework for safe feedback motion planning from pixels using robust model predictive control (MPC) in learned latent world models. Our approach trains an action-conditioned joint-embedding world model with compact Markovian latent states, enabling efficient gradient-based trajectory optimization through learned latent dynamics. To enforce safety for the true system despite imperfect latent predictions, we inform a GPU-accelerated system level synthesis (SLS) robust MPC scheme with conformal prediction to obtain calibrated latent error bounds and robust latent-space constraint sets. We further learn and conformalize a latent constraint checker, allowing the SLS planner to impose probabilistic safety constraints during closed-loop execution. We evaluate our method on vision-based control tasks, where it improves both goal-reaching performance and safety over latent world-model and safe-planning baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 52 of its 62 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

52 of 62 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Training Agents Inside of Scalable World Models29 Sep
  2. 2025Safety Certification in the Latent space using Control Barrier Functions and World Models18 Jul
  3. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures1 May
  4. 2025Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis2 Feb
  5. 2024Efficient World Models with Context-Aware Tokenization27 Jun
  6. 2024Diffusion for World Modeling: Visual Details Matter in Atari20 May
  7. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  8. 2023Mastering Diverse Domains through World Models10 Jan
  9. 2022Transformers are Sample-Efficient World Models1 Sep
  10. 2020Mastering Atari with Discrete World Models5 Oct
  11. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  12. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
1 more
  1. 2018World Models27 Mar

Other references · 39

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  3. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  4. 2018DeepMind Control Suitecited by 956
  5. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  6. 2024Genie: Generative Interactive Environmentscited by 747
  7. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  8. 2019Conformal Prediction Under Covariate Shiftcited by 519
  9. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  10. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  11. 2019Control Barrier Functions: Theory and Applicationscited by 372
  12. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
27 more
  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 273
  2. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
  3. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  4. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  5. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
  6. 2020Learning Control Barrier Functions from Expert Demonstrationscited by 92
  7. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 89
  8. 2015Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Designcited by 39
  9. 2026Temporal Straightening for Latent Planningcited by 38
  10. 2020Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functionscited by 33
  11. 2022Lyapunov Density Models: Constraining Distribution Shift in Learning-Based Controlcited by 28
  12. 2020Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constantscited by 25
  13. 2025Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distributioncited by 25
  14. 2021Learning Robust Output Control Barrier Functions from Safe Expert Demonstrationscited by 20
  15. 2022Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theorycited by 19
  16. 2023Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matchingcited by 17
  17. 2023In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution Statescited by 16
  18. 2021Model Error Propagation via Learned Contraction Metrics for Safe Feedback Motion Planning of Unknown Systemscited by 13
  19. 2022Statistical Safety and Robustness Guarantees for Feedback Motion Planning of Unknown Underactuated Stochastic Systemscited by 11
  20. 2025How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraintscited by 9
  21. 2023How Safe Am I Given What I See? Calibrated Prediction of Safety Chances for Image-Controlled Autonomycited by 8
  22. 2023Synthesizing Stable Reduced-Order Visuomotor Policies for Nonlinear Systems via Sums-of-Squares Optimizationcited by 7
  23. 2026Safety Beyond the Training Data: Robust Out-of-Distribution MPC via Conformalized System Level Synthesiscited by 7
  24. 2026Safe Large-Scale Robust Nonlinear MPC in Milliseconds via Reachability-Constrained System Level Synthesis on the GPUcited by 7
  25. 2025Accelerating Residual Reinforcement Learning with Uncertainty Estimationcited by 6
  26. 2026VISION-SLS: Safe Perception-Based Control from Learned Visual Representations via System Level Synthesiscited by 3
  27. 2025Uncertainty-aware Planning with Inaccurate Models for Robotized Liquid Handlingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026GPU-Parallel Linearization Error Bounds for Real-Time Robust Optimal Control of Nonlinear and Neural Network Dynamicscited by 0
  2. 2026No Free Checker: A Survey of Verifiers for Robot Policiescited by 0
  3. 2026Preserving Unstable Modes Through Inverse Dynamics in JEPA World Modelscited by 0
  4. 2026Modeling Latent Disturbances for Robust Decision-Making in World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025How Well Do Latent World Models Understand Partially Observable Safety Constraints?
  2. 2025AnySafe: Adapting Latent Safety Filters at Runtime via Safety Constraint Parameterization in the Latent Space
  3. 2026LeFlow: Generative Latent Flow Planning for World Models
  4. 2026Learning from World Feedback: Why Model Uncertainty Fails as a Risk Signal in Model-Based RL
  5. 2025How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy
  6. 2021Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models
  7. 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
  8. 2025Deterministic World Models for Closed-loop Reachability Analysis of End-to-End Vision-based Control