# VISION-SLS: Safe Perception-Based Control from Learned Visual Representations via System Level Synthesis

arXiv: https://arxiv.org/abs/2604.24894 · HTML: https://mlsanity.com/paper/2604.24894/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 30 of 31)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Efficient Neural Network Robustness Certification with General Activation Functions](https://mlsanity.com/paper/1811.00866/) (2018)
- [Learning Invariant Representations for Reinforcement Learning without Reconstruction](https://mlsanity.com/paper/2006.10742/) (2020)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [OSQP: An Operator Splitting Solver for Quadratic Programs](https://mlsanity.com/paper/1711.08013/) (2017)
- [ViNG: Learning Open-World Navigation with Visual Goals](https://mlsanity.com/paper/2012.09812/) (2020)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Robust Locally-Linear Controllable Embedding](https://mlsanity.com/paper/1710.05373/) (2017)
- [Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions](https://mlsanity.com/paper/2010.16001/) (2020)
- [Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constants](https://mlsanity.com/paper/2010.08993/) (2020)
- [Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theory](https://mlsanity.com/paper/2206.06553/) (2022)
- [GPU Acceleration of ADMM for Large-Scale Quadratic Programming](https://mlsanity.com/paper/1912.04263/) (2019)
- [Safe Perception-Based Control under Stochastic Sensor Uncertainty using Conformal Prediction](https://mlsanity.com/paper/2304.00194/) (2023)
- [Model Error Propagation via Learned Contraction Metrics for Safe Feedback Motion Planning of Unknown Systems](https://mlsanity.com/paper/2104.08695/) (2021)
- [Certainty Equivalent Perception-Based Control](https://mlsanity.com/paper/2008.12332/) (2020)
- [Statistical Safety and Robustness Guarantees for Feedback Motion Planning of Unknown Underactuated Stochastic Systems](https://mlsanity.com/paper/2212.06874/) (2022)
- [Robust Tracking with Model Mismatch for Fast and Safe Planning: an SOS Optimization Approach](https://mlsanity.com/paper/1808.00649/) (2018)
- [Compositional Learning-based Planning for Vision POMDPs](https://mlsanity.com/paper/2112.09456/) (2021)
- [Synthesizing Stable Reduced-Order Visuomotor Policies for Nonlinear Systems via Sums-of-Squares Optimization](https://mlsanity.com/paper/2304.12405/) (2023)
- [Safety Beyond the Training Data: Robust Out-of-Distribution MPC via Conformalized System Level Synthesis](https://mlsanity.com/paper/2602.12047/) (2026)
- [Safe Large-Scale Robust Nonlinear MPC in Milliseconds via Reachability-Constrained System Level Synthesis on the GPU](https://mlsanity.com/paper/2604.07644/) (2026)
- [Fast System Level Synthesis: Robust Model Predictive Control using Riccati Recursions](https://mlsanity.com/paper/2401.13762/) (2024)
- [Robustly Constrained Dynamic Games for Uncertain Nonlinear Dynamics](https://mlsanity.com/paper/2509.16826/) (2025)
- [A System Level Approach to Tube-based Model Predictive Control](https://mlsanity.com/paper/2103.02460/) (2021)
- [Guaranteed Robust Nonlinear MPC via Disturbance Feedback](https://mlsanity.com/paper/2509.18760/) (2025)
- [Safe Control of Partially-Observed Linear Time-Varying Systems with Minimal Worst-Case Dynamic Regret](https://mlsanity.com/paper/2208.08929/) (2022)

## Built on by (most cited first, 3 of 3)

- [Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC](https://mlsanity.com/paper/2606.15594/) (2026)
- [Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulation](https://mlsanity.com/paper/2606.14188/) (2026)
- [GPU-Parallel Linearization Error Bounds for Real-Time Robust Optimal Control of Nonlinear and Neural Network Dynamics](https://mlsanity.com/paper/2607.01203/) (2026)

## Abstract (arXiv)

We propose VISION-SLS, a method for nonlinear output-feedback control from high-resolution RGB images which provides robust constraint satisfaction guarantees under calibrated uncertainty bounds despite partial observability, sensor noise, and nonlinear dynamics. To enable scalability while retaining guarantees, we propose: (i) a learned low-dimensional observation map from pretrained visual features with state-dependent error bounds, and (ii) a causal affine time-varying output-feedback policy optimized via System Level Synthesis (SLS). We develop a scalable, novel solver for the resulting nonconvex program that leverages sequential convex programming coupled with efficient Riccati recursions. On two simulated visuomotor tasks (a 4D car and a 10D quadrotor) with >= 512 x 512 pixels and a 59D humanoid task with partial observability, our method enables safe, information-gathering behavior that reduces uncertainty while guaranteeing constraint satisfaction with empirically-calibrated error bounds. We also validate our method on hardware, safely controlling a ground vehicle from onboard images, outperforming baselines in safety rate and solve times. Together, these results show that learned visual abstractions coupled with an efficient solver make SLS-based safe visuomotor output-feedback practical at scale. The code implementation of our method is available at https://github.com/trustworthyrobotics/VISION-SLS.