mlsanityData to 13 Oct 2026

VISION-SLS: Safe Perception-Based Control from Learned Visual Representations via System Level Synthesis

arXiv 2604.24894 ↗First posted 27 Apr 2026cs.RO cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We propose VISION-SLS, a method for nonlinear output-feedback control from high-resolution RGB images which provides robust constraint satisfaction guarantees under calibrated uncertainty bounds despite partial observability, sensor noise, and nonlinear dynamics. To enable scalability while retaining guarantees, we propose: (i) a learned low-dimensional observation map from pretrained visual features with state-dependent error bounds, and (ii) a causal affine time-varying output-feedback policy optimized via System Level Synthesis (SLS). We develop a scalable, novel solver for the resulting nonconvex program that leverages sequential convex programming coupled with efficient Riccati recursions. On two simulated visuomotor tasks (a 4D car and a 10D quadrotor) with >= 512 x 512 pixels and a 59D humanoid task with partial observability, our method enables safe, information-gathering behavior that reduces uncertainty while guaranteeing constraint satisfaction with empirically-calibrated error bounds. We also validate our method on hardware, safely controlling a ground vehicle from onboard images, outperforming baselines in safety rate and solve times. Together, these results show that learned visual abstractions coupled with an efficient solver make SLS-based safe visuomotor output-feedback practical at scale. The code implementation of our method is available at https://github.com/trustworthyrobotics/VISION-SLS.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 31 of its 51 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

31 of 51 references.

References · 31

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,760
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 850
  4. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 442
  5. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 431
  6. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 369
  7. 2021Deep ViT Features as Dense Visual Descriptorscited by 329
  8. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 270
  9. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 156
  10. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 78
  11. 2019acados: a modular open-source framework for fast embedded optimal controlcited by 61
  12. 2017Robust Locally-Linear Controllable Embeddingcited by 37
19 more
  1. 2020Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functionscited by 33
  2. 2020Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constantscited by 25
  3. 2022Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theorycited by 19
  4. 2019GPU Acceleration of ADMM for Large-Scale Quadratic Programmingcited by 15
  5. 2023Safe Perception-Based Control under Stochastic Sensor Uncertainty using Conformal Predictioncited by 15
  6. 2021Model Error Propagation via Learned Contraction Metrics for Safe Feedback Motion Planning of Unknown Systemscited by 13
  7. 2020Certainty Equivalent Perception-Based Controlcited by 11
  8. 2022Statistical Safety and Robustness Guarantees for Feedback Motion Planning of Unknown Underactuated Stochastic Systemscited by 11
  9. 2018Robust Tracking with Model Mismatch for Fast and Safe Planning: an SOS Optimization Approachcited by 8
  10. 2021Compositional Learning-based Planning for Vision POMDPscited by 8
  11. 2023Synthesizing Stable Reduced-Order Visuomotor Policies for Nonlinear Systems via Sums-of-Squares Optimizationcited by 7
  12. 2026Safety Beyond the Training Data: Robust Out-of-Distribution MPC via Conformalized System Level Synthesiscited by 7
  13. 2026Safe Large-Scale Robust Nonlinear MPC in Milliseconds via Reachability-Constrained System Level Synthesis on the GPUcited by 7
  14. 2024Fast System Level Synthesis: Robust Model Predictive Control using Riccati Recursionscited by 5
  15. 2025Robustly Constrained Dynamic Games for Uncertain Nonlinear Dynamicscited by 5
  16. 2021A System Level Approach to Tube-based Model Predictive Controlcited by 4
  17. 2025Guaranteed Robust Nonlinear MPC via Disturbance Feedbackcited by 3
  18. 2022Safe Control of Partially-Observed Linear Time-Varying Systems with Minimal Worst-Case Dynamic Regretcited by 2
  19. 2022A dual-control effect preserving formulation for nonlinear output-feedback stochastic model predictive control with constraintscited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPCcited by 4
  2. 2026Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulationcited by 3
  3. 2026GPU-Parallel Linearization Error Bounds for Real-Time Robust Optimal Control of Nonlinear and Neural Network Dynamicscited by 0