mlsanityData to 7 Oct 2026

Sample-Efficient Safety Assurances using Conformal Prediction

arXiv 2109.14082 ↗First posted 28 Sep 2021cs.RO cs.LGcited by 41

Not on the map. Why?

Abstract (arXiv)

When deploying machine learning models in high-stakes robotics applications, the ability to detect unsafe situations is crucial. Early warning systems can provide alerts when an unsafe situation is imminent (in the absence of corrective action). To reliably improve safety, these warning systems should have a provable false negative rate; i.e. of the situations that are unsafe, fewer than ε will occur without an alert. In this work, we present a framework that combines a statistical inference technique known as conformal prediction with a simulator of robot/environment dynamics, in order to tune warning systems to provably achieve an ε false negative rate using as few as 1/ε data points. We apply our framework to a driver warning system and a robotic grasping application, and empirically demonstrate guaranteed false negative rate while also observing low false detection (positive) rate.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 16 of its 50 references that are in the map. First posted on arXiv 28 Sep 2021.

Builds on

16 of 50 references.

References · 16

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2019Conformal Prediction Under Covariate Shiftcited by 511
  3. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  4. 2020Performative Predictioncited by 318
  5. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  6. 2020Robust Validation: Confident Predictions Even When Distributions Shiftcited by 82
  7. 2016Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challengecited by 70
  8. 2021Online Multivalid Learning: Means, Moments, and Prediction Intervalscited by 48
  9. 2021Improving Conditional Coverage via Orthogonal Quantile Regressioncited by 48
  10. 2020Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systemscited by 46
  11. 2016Analysis and Observations from the First Amazon Picking Challengecited by 41
  12. 2017Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learningcited by 15
4 more
  1. 2016Team Delft's Robot Winner of the Amazon Picking Challenge 2016cited by 10
  2. 2020Reactive motion planning with probabilistic safety guaranteescited by 10
  3. 2016A Summary of Team MIT's Approach to the Amazon Picking Challenge 2015cited by 3
  4. 2016Guaranteed Model-Based Fault Detection in Cyber-Physical Systems: A Model Invalidation Approachcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 13; 2024: 10; 2025: 4; 2026: 9.

41 papers.

Sort:

By area:

  1. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  2. 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 67
  3. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 63
  4. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 53
  5. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 46
  6. 2022A System-Level View on Out-of-Distribution Data in Roboticscited by 33
  7. 2023Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagationcited by 33
  8. 2023Conformal Prediction Regions for Time Series using Linear Complementarity Programmingcited by 33
  9. 2022Copula Conformal Prediction for Multi-step Time Series Forecastingcited by 28
  10. 2024Adaptive Bounding Box Uncertainties via Two-Step Conformal Predictioncited by 23
  11. 2024How Generalizable Is My Behavior Cloning Policy? A Statistical Approach to Trustworthy Performance Evaluationcited by 14
  12. 2023Distributionally Robust Statistical Verification with Imprecise Neural Networkscited by 12
29 more
  1. 2022Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Surveycited by 11
  2. 2024Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedbackcited by 11
  3. 2023Data-Driven Reachability Analysis of Stochastic Dynamical Systems with Conformal Inferencecited by 9
  4. 2024Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusioncited by 9
  5. 2023Conformal Policy Learning for Sensorimotor Control Under Distribution Shiftscited by 5
  6. 2023Bridging Dimensions: Confident Reachability for High-Dimensional Controllerscited by 5
  7. 2024Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgerycited by 5
  8. 2025SafeMimic: Towards Safe and Autonomous Human-to-Robot Imitation for Mobile Manipulationcited by 5
  9. 2026Conformal Policy Controlcited by 5
  10. 2023Multi-Modal Conformal Prediction Regions with Simple Structures by Optimizing Convex Shape Templatescited by 4
  11. 2024Recursively Feasible Shrinking-Horizon MPC in Dynamic Environments with Conformal Prediction Guaranteescited by 4
  12. 2024Conformalized Teleoperation: Confidently Mapping Human Inputs to High-Dimensional Robot Actionscited by 4
  13. 2024Statistical Reachability Analysis of Stochastic Cyber-Physical Systems under Distribution Shiftcited by 4
  14. 2023Out of Distribution Detection via Domain-Informed Gaussian Process State Space Modelscited by 3
  15. 2022Probabilistic Symmetry for Multi-Agent Dynamicscited by 2
  16. 2023Air Traffic Controller Workload Level Prediction using Conformalized Dynamical Graph Learningcited by 2
  17. 2025Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policiescited by 2
  18. 2026CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Predictioncited by 2
  19. 2026Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulatorscited by 2
  20. 2025Uncertainty Guarantees on Automated Precision Weeding using Conformal Predictioncited by 1
  21. 2025Reliable Classification with Conformal Learning and Interval-Type 2 Fuzzy Setscited by 1
  22. 2026Spatiotemporal Robustness of Temporal Logic Tasks using Multi-Objective Reasoningcited by 1
  23. 2026On Optimal Data Splitting for Split Conformal Predictioncited by 1
  24. 2026CURA: Certified Runtime Alarms for Computer-Use Agentscited by 1
  25. 2023Enterprise Disk Drive Scrubbing Based on Mondrian Conformal Predictorscited by 0
  26. 2023CONFIDERAI: a novel CONFormal Interpretable-by-Design score function for Explainable and Reliable Artificial Intelligencecited by 0
  27. 2026Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spacescited by 0
  28. 2026Model-Based Reinforcement Learning for Heterogeneous Multi-Robot Task Assignment Under Distribution Shiftscited by 0
  29. 2026A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantificationcited by 0