mlsanityData to 7 Oct 2026

A System-Level View on Out-of-Distribution Data in Robotics

arXiv 2212.14020 ↗First posted 28 Dec 2022cs.RO cs.LG cs.SYcited by 33

Not on the map. Why?

Abstract (arXiv)

When testing conditions differ from those represented in training data, so-called out-of-distribution (OOD) inputs can mar the reliability of learned components in the modern robot autonomy stack. Therefore, coping with OOD data is an important challenge on the path towards trustworthy learning-enabled open-world autonomy. In this paper, we aim to demystify the topic of OOD data and its associated challenges in the context of data-driven robotic systems, drawing connections to emerging paradigms in the ML community that study the effect of OOD data on learned models in isolation. We argue that as roboticists, we should reason about the overall textitsystem-level competence of a robot as it operates in OOD conditions. We highlight key research questions around this system-level view of OOD problems to guide future research toward safe and reliable learning-enabled autonomy.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 42 of its 63 references that are in the map. First posted on arXiv 28 Dec 2022.

Builds on

42 of 63 references.

References · 42

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  10. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  11. 2019Invariant Risk Minimizationcited by 1,832
  12. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
30 more
  1. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  4. 2020In Search of Lost Domain Generalizationcited by 1,042
  5. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  6. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  7. 2021Domain Generalization: A Surveycited by 760
  8. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  9. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
  10. 2015Policy Distillationcited by 414
  11. 2018A Survey of Unsupervised Deep Domain Adaptationcited by 361
  12. 2019Deep Evidential Regressioncited by 354
  13. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  14. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  15. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  16. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  17. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  18. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  19. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  20. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  21. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  22. 2021Sample-Efficient Safety Assurances using Conformal Predictioncited by 41
  23. 2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 38
  24. 2021Tracking the risk of a deployed model and detecting harmful distribution shiftscited by 38
  25. 2016Towards Verified Artificial Intelligencecited by 36
  26. 2021Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networkscited by 32
  27. 2021Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trendscited by 26
  28. 2020On Infusing Reachability-Based Safety Assurance within Planning Frameworks for Human-Robot Vehicle Interactionscited by 24
  29. 2022Out-Of-Distribution Detection Is Not All You Needcited by 23
  30. 2022Failure Prediction with Statistical Guarantees for Vision-Based Robot Controlcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 6; 2025: 15; 2026: 6.

33 papers.

Sort:

By area:

  1. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 74
  2. 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 67
  3. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 63
  4. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
  5. 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Drivingcited by 55
  6. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  7. 2025Failure Prediction at Runtime for Generative Robot Policiescited by 36
  8. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 33
  9. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  10. 2023Distributionally Robust Statistical Verification with Imprecise Neural Networkscited by 12
  11. 2025Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoningcited by 10
  12. 2025Mobi-π: Mobilizing Your Robot Learning Policycited by 9
21 more
  1. 2025Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Modelscited by 8
  2. 2023Topology-Matching Normalizing Flows for Out-of-Distribution Detection in Robot Learningcited by 7
  3. 2025Real-Time Operator Takeover for Visuomotor Diffusion Policy Trainingcited by 5
  4. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  5. 2023Detecting and Mitigating System-Level Anomalies of Vision-Based Controllerscited by 4
  6. 2023Out of Distribution Detection via Domain-Informed Gaussian Process State Space Modelscited by 3
  7. 2024First, Learn What You Don't Know: Active Information Gathering for Driving at the Limits of Handlingcited by 3
  8. 2025Vision Foundation Model Embedding-Based Semantic Anomaly Detectioncited by 3
  9. 2023Density-based Feasibility Learning with Normalizing Flows for Introspective Robotic Assemblycited by 2
  10. 2024Diagnostic Runtime Monitoring with Martingalescited by 2
  11. 2025Scan, Materialize, Simulate: A Generalizable Framework for Physically Grounded Robot Planningcited by 2
  12. 2025Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systemscited by 2
  13. 2026Early Warning Signals for OpenVLA Failure under Visual Distribution Shiftcited by 2
  14. 2026SAFECAST: Robust Failure Detection for VLA Policies with Contrast-Set Training and Calibrationcited by 2
  15. 2025Conformal Safety Monitoring for Flight Testing: A Case Study in Data-Driven Safety Learningcited by 1
  16. 2026VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Modelscited by 1
  17. 2025Deceptive Risk Minimization: Out-of-Distribution Generalization by Deceiving Distribution Shift Detectorscited by 0
  18. 2025Foundation models on the bridge: Semantic hazard detection and safety maneuvers for maritime autonomy with vision-language modelscited by 0
  19. 2026Sensitivity Shaping for Latent Modelingcited by 0
  20. 2026Towards Trustworthy Embodied Intelligence: A Systems Framework and Graded Trustworthiness Levelscited by 0
  21. 2026Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Modelscited by 0