# Competence-Aware Path Planning via Introspective Perception

arXiv: https://arxiv.org/abs/2109.13974 · HTML: https://mlsanity.com/paper/2109.13974/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 8 of 8)

- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Introspective Perception: Learning to Predict Failures in Vision Systems](https://mlsanity.com/paper/1607.08665/) (2016)
- [The STRANDS Project: Long-Term Autonomy in Everyday Environments](https://mlsanity.com/paper/1604.04384/) (2016)
- [Perception-aware Path Planning](https://mlsanity.com/paper/1605.04151/) (2016)
- [Risk-Aware Motion Planning in Partially Known Environments](https://mlsanity.com/paper/2109.11287/) (2021)
- [Learning to Optimize Autonomy in Competence-Aware Systems](https://mlsanity.com/paper/2003.07745/) (2020)
- [IVOA: Introspective Vision for Obstacle Avoidance](https://mlsanity.com/paper/1903.01028/) (2019)
- [IV-SLAM: Introspective Vision for Simultaneous Localization and Mapping](https://mlsanity.com/paper/2008.02760/) (2020)

## Abstract (arXiv)

Robots deployed in the real world over extended periods of time need to reason about unexpected failures, learn to predict them, and to proactively take actions to avoid future failures. Existing approaches for competence-aware planning are either model-based, requiring explicit enumeration of known failure modes, or purely statistical, using state- and location-specific failure statistics to infer competence. We instead propose a structured model-free approach to competence-aware planning by reasoning about plan execution failures due to errors in perception, without requiring a priori enumeration of failure sources or requiring location-specific failure statistics. We introduce competence-aware path planning via introspective perception (CPIP), a Bayesian framework to iteratively learn and exploit task-level competence in novel deployment environments. CPIP factorizes the competence-aware planning problem into two components. First, perception errors are learned in a model-free and location-agnostic setting via introspective perception prior to deployment in novel environments. Second, during actual deployments, the prediction of task-level failures is learned in a context-aware setting. Experiments in a simulation show that the proposed CPIP approach outperforms the frequentist baseline in multiple mobile robot tasks, and is further validated via real robot experiments in an environment with perceptually challenging obstacles and terrain.