# Aleatoric Uncertainty from AI-based 6D Object Pose Predictors for Object-relative State Estimation

arXiv: https://arxiv.org/abs/2509.01583 · HTML: https://mlsanity.com/paper/2509.01583/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Scaled-YOLOv4: Scaling Cross Stage Partial Network](https://mlsanity.com/paper/2011.08036/) (2020)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [CoordiNet: uncertainty-aware pose regressor for reliable vehicle localization](https://mlsanity.com/paper/2103.10796/) (2021)
- [Quaternion kinematics for the error-state Kalman filter](https://mlsanity.com/paper/1711.02508/) (2017)
- [Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2205.01823/) (2022)
- [Multivariate Uncertainty in Deep Learning](https://mlsanity.com/paper/1910.14215/) (2019)
- [SpyroPose: SE(3) Pyramids for Object Pose Distribution Estimation](https://mlsanity.com/paper/2303.05308/) (2023)
- [PoET: Pose Estimation Transformer for Single-View, Multi-Object 6D Pose Estimation](https://mlsanity.com/paper/2211.14125/) (2022)
- [Uncertainty Quantification with Deep Ensembles for 6D Object Pose Estimation](https://mlsanity.com/paper/2403.07741/) (2024)
- [Probabilistic Regression of Rotations using Quaternion Averaging and a Deep Multi-Headed Network](https://mlsanity.com/paper/1904.03182/) (2019)
- [NVINS: Robust Visual Inertial Navigation Fused with NeRF-augmented Camera Pose Regressor and Uncertainty Quantification](https://mlsanity.com/paper/2404.01400/) (2024)
- [AIVIO: Closed-loop, Object-relative Navigation of UAVs with AI-aided Visual Inertial Odometry](https://mlsanity.com/paper/2410.05996/) (2024)
- [A Probabilistic Rotation Representation for Symmetric Shapes With an Efficiently Computable Bingham Loss Function](https://mlsanity.com/paper/2305.18947/) (2023)
- [AI-Based Multi-Object Relative State Estimation with Self-Calibration Capabilities](https://mlsanity.com/paper/2303.00371/) (2023)

## Abstract (arXiv)

Deep Learning (DL) has become essential in various robotics applications due to excelling at processing raw sensory data to extract task specific information from semantic objects. For example, vision-based object-relative navigation relies on a DL-based 6D object pose predictor to provide the relative pose between the object and the robot as measurements to the robot's state estimator. Accurately knowing the uncertainty inherent in such Deep Neural Network (DNN) based measurements is essential for probabilistic state estimators subsequently guiding the robot's tasks. Thus, in this letter, we show that we can extend any existing DL-based object-relative pose predictor for aleatoric uncertainty inference simply by including two multi-layer perceptrons detached from the translational and rotational part of the DL predictor. This allows for efficient training while freezing the existing pre-trained predictor. We then use the inferred 6D pose and its uncertainty as a measurement and corresponding noise covariance matrix in an extended Kalman filter (EKF). Our approach induces minimal computational overhead such that the state estimator can be deployed on edge devices while benefiting from the dynamically inferred measurement uncertainty. This increases the performance of the object-relative state estimation task compared to a fix-covariance approach. We conduct evaluations on synthetic data and real-world data to underline the benefits of aleatoric uncertainty inference for the object-relative state estimation task.