mlsanityData to 14 Oct 2026

Safe Perception-Based Control under Stochastic Sensor Uncertainty using Conformal Prediction

arXiv 2304.00194 ↗First posted 1 Apr 2023eess.SY cs.LG cs.ROcited by 16

Not on the map. Why?

Abstract (arXiv)

We consider perception-based control using state estimates that are obtained from high-dimensional sensor measurements via learning-enabled perception maps. However, these perception maps are not perfect and result in state estimation errors that can lead to unsafe system behavior. Stochastic sensor noise can make matters worse and result in estimation errors that follow unknown distributions. We propose a perception-based control framework that i) quantifies estimation uncertainty of perception maps, and ii) integrates these uncertainty representations into the control design. To do so, we use conformal prediction to compute valid state estimation regions, which are sets that contain the unknown state with high probability. We then devise a sampled-data controller for continuous-time systems based on the notion of measurement robust control barrier functions. Our controller uses idea from self-triggered control and enables us to avoid using stochastic calculus. Our framework is agnostic to the choice of the perception map, independent of the noise distribution, and to the best of our knowledge the first to provide probabilistic safety guarantees in such a setting. We demonstrate the effectiveness of our proposed perception-based controller for a LiDAR-enabled F1/10th car.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 1 Apr 2023.

Builds on

31 of 49 references.

References · 31

  1. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  2. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  3. 2019Conformal Prediction Under Covariate Shiftcited by 519
  4. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
  5. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  6. 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 119
  7. 2022Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imagingcited by 107
  8. 2022Adaptive Conformal Prediction for Motion Planning among Dynamic Agentscited by 53
  9. 2020Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functionscited by 33
  10. 2022Learning Safe, Generalizable Perception-based Hybrid Control with Certificatescited by 24
  11. 2018PAC-Bayes Control: Learning Policies that Provably Generalize to Novel Environmentscited by 22
  12. 2019Robust Guarantees for Perception-Based Controlcited by 20
19 more
  1. 2021Learning Robust Output Control Barrier Functions from Safe Expert Demonstrationscited by 20
  2. 2022Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theorycited by 19
  3. 2019Case Study: Verifying the Safety of an Autonomous Racing Car with a Neural Network Controllercited by 17
  4. 2020Probably Approximately Correct Vision-Based Planning using Motion Primitivescited by 16
  5. 2020Control Barrier Functions for Stochastic Systemscited by 14
  6. 2020Perception-Based Temporal Logic Planning in Uncertain Semantic Mapscited by 12
  7. 2022A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racingcited by 12
  8. 2021Measurement-Robust Control Barrier Functions: Certainty in Safety with Uncertainty in Statecited by 8
  9. 2022Differentiable Safe Controller Design through Control Barrier Functionscited by 8
  10. 2021Neural Predictive Monitoring under Partial Observabilitycited by 7
  11. 2022Self-Supervised Online Learning for Safety-Critical Control using Stereo Visioncited by 6
  12. 2021Control Barrier Function Meets Interval Analysis: Safety-Critical Control with Measurement and Actuation Uncertaintiescited by 3
  13. 2022Safe and Robust Observer-Controller Synthesis using Control Barrier Functionscited by 3
  14. 2021Feedback from Pixels: Output Regulation via Learning-Based Scene View Synthesiscited by 2
  15. 2021Observer-based Control Barrier Functions for Safety Critical Systemscited by 2
  16. 2022Online Optimization of Dynamical Systems with Deep Learning Perceptioncited by 1
  17. 2022Attacks on Perception-Based Control Systems: Modeling and Fundamental Limitscited by 1
  18. 2022Learning Certifiably Robust Controllers Using Fragile Perceptioncited by 1
  19. 2022Perception-Based Sampled-Data Optimization of Dynamical Systemscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 6; 2026: 6.

16 papers.

Sort:

By area:

  1. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 173
  2. 2024MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertaintycited by 32
  3. 2024SoNIC: Safe Social Navigation with Adaptive Conformal Inference and Constrained Reinforcement Learningcited by 9
  4. 2023How Safe Am I Given What I See? Calibrated Prediction of Safety Chances for Image-Controlled Autonomycited by 8
  5. 2025Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handlingcited by 6
  6. 2025Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstractioncited by 4
  7. 2024Conformal Off-Policy Prediction for Multi-Agent Systemscited by 3
  8. 2025Conformal Safety Shielding for Imperfect-Perception Agentscited by 3
  9. 2026VISION-SLS: Safe Perception-Based Control from Learned Visual Representations via System Level Synthesiscited by 3
  10. 2026Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulationcited by 3
  11. 2025DoublyAware: Dual Planning and Policy Awareness for Temporal Difference Learning in Humanoid Locomotioncited by 1
  12. 2025How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomycited by 1
4 more
  1. 2026Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spacescited by 1
  2. 2026Statistical Contraction for Chance-Constrained Trajectory Optimization of Non-Gaussian Stochastic Systemscited by 0
  3. 2026Differentiable Optimization Layered Safety-Critical Control for Risk-Aware Navigation via Conformal Predictioncited by 0
  4. 2026A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantificationcited by 0