# Learning Visual Information Utility with PIXER

arXiv: https://arxiv.org/abs/2409.13151 · HTML: https://mlsanity.com/paper/2409.13151/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 23 of 23)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [SiLK -- Simple Learned Keypoints](https://mlsanity.com/paper/2304.06194/) (2023)
- [Estimating Epistemic and Aleatoric Uncertainty with a Single Model](https://mlsanity.com/paper/2402.03478/) (2024)
- [Toward Geometric Deep SLAM](https://mlsanity.com/paper/1707.07410/) (2017)
- [GLAMpoints: Greedily Learned Accurate Match points](https://mlsanity.com/paper/1908.06812/) (2019)
- [Towards Robust Robot 3D Perception in Urban Environments: The UT Campus Object Dataset](https://mlsanity.com/paper/2309.13549/) (2023)
- [UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor](https://mlsanity.com/paper/1907.04011/) (2019)
- [UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM](https://mlsanity.com/paper/2306.11048/) (2023)
- [On the Estimation of Image-matching Uncertainty in Visual Place Recognition](https://mlsanity.com/paper/2404.00546/) (2024)
- [Lightweight, Uncertainty-Aware Conformalized Visual Odometry](https://mlsanity.com/paper/2303.02207/) (2023)
- [Self-Improving Visual Odometry](https://mlsanity.com/paper/1812.03245/) (2018)
- [Tightly-coupled Fusion of Global Positional Measurements in Optimization-based Visual-Inertial Odometry](https://mlsanity.com/paper/2003.04159/) (2020)
- [CUAHN-VIO: Content-and-Uncertainty-Aware Homography Network for Visual-Inertial Odometry](https://mlsanity.com/paper/2208.13935/) (2022)
- [Network Uncertainty Informed Semantic Feature Selection for Visual SLAM](https://mlsanity.com/paper/1811.11946/) (2018)
- [ComBiNet: Compact Convolutional Bayesian Neural Network for Image Segmentation](https://mlsanity.com/paper/2104.06957/) (2021)
- [l-dyno: framework to learn consistent visual features using robot's motion](https://mlsanity.com/paper/2310.06249/) (2023)
- [On Designing Consistent Covariance Recovery from a Deep Learning Visual Odometry Engine](https://mlsanity.com/paper/2403.13170/) (2024)

## Abstract (arXiv)

Accurate feature detection is fundamental for various computer vision tasks, including autonomous robotics, 3D reconstruction, medical imaging, and remote sensing. Despite advancements in enhancing the robustness of visual features, no existing method measures the utility of visual information before processing by specific feature-type algorithms. To address this gap, we introduce PIXER and the concept of "Featureness," which reflects the inherent interest and reliability of visual information for robust recognition, independent of any specific feature type. Leveraging a generalization on Bayesian learning, our approach quantifies both the probability and uncertainty of a pixel's contribution to robust visual utility in a single-shot process, avoiding costly operations such as Monte Carlo sampling and permitting customizable featureness definitions adaptable to a wide range of applications. We evaluate PIXER on visual odometry with featureness selectivity, achieving an average of 31% improvement in RMSE trajectory with 49% fewer features.