# The SET Perceptual Factors Framework: Towards Assured Perception for Autonomous Systems

arXiv: https://arxiv.org/abs/2508.10798 · HTML: https://mlsanity.com/paper/2508.10798/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 48 references that are in the map.
- First posted on arXiv 14 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey](https://mlsanity.com/paper/2112.08936/) (2021)
- [DAWN: Vehicle Detection in Adverse Weather Nature Dataset](https://mlsanity.com/paper/2008.05402/) (2020)
- [WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models](https://mlsanity.com/paper/2305.07528/) (2023)
- [NOD: Taking a Closer Look at Detection under Extreme Low-Light Conditions with Night Object Detection Dataset](https://mlsanity.com/paper/2110.10364/) (2021)
- [GLARE: A Dataset for Traffic Sign Detection in Sun Glare](https://mlsanity.com/paper/2209.08716/) (2022)
- [GEM: Glare or Gloom, I Can Still See You -- End-to-End Multimodal Object Detection](https://mlsanity.com/paper/2102.12319/) (2021)
- [IV-SLAM: Introspective Vision for Simultaneous Localization and Mapping](https://mlsanity.com/paper/2008.02760/) (2020)
- [Modelling Observation Correlations for Active Exploration and Robust Object Detection](https://mlsanity.com/paper/1401.4612/) (2014)
- [Robust Glare Detection: Review, Analysis, and Dataset Release](https://mlsanity.com/paper/2110.06006/) (2021)
- [Perceptual Factors for Environmental Modeling in Robotic Active Perception](https://mlsanity.com/paper/2309.10620/) (2023)

## Abstract (arXiv)

Future autonomous systems promise significant societal benefits, yet their deployment raises concerns about safety and trustworthiness. A key concern is assuring the reliability of robot perception, as perception seeds safe decision-making. Failures in perception are often due to complex yet common environmental factors and can lead to accidents that erode public trust. To address this concern, we introduce the SET (Self, Environment, and Target) Perceptual Factors Framework. We designed the framework to systematically analyze how factors such as weather, occlusion, or sensor limitations negatively impact perception. To achieve this, the framework employs SET State Trees to categorize where such factors originate and SET Factor Trees to model how these sources and factors impact perceptual tasks like object detection or pose estimation. Next, we develop Perceptual Factor Models using both trees to quantify the uncertainty for a given task. Our framework aims to promote rigorous safety assurances and cultivate greater public understanding and trust in autonomous systems by offering a transparent and standardized method for identifying, modeling, and communicating perceptual risks.