# A Neurosymbolic Approach to Adaptive Feature Extraction in SLAM

arXiv: https://arxiv.org/abs/2407.06889 · HTML: https://mlsanity.com/paper/2407.06889/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 8 of 8)

- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Neuro-Symbolic Program Synthesis](https://mlsanity.com/paper/1611.01855/) (2016)
- [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023)
- [A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence](https://mlsanity.com/paper/2006.12567/) (2020)
- [Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images](https://mlsanity.com/paper/1710.02726/) (2017)
- [Neurosymbolic Models for Computer Graphics](https://mlsanity.com/paper/2304.10320/) (2023)
- [Enhancing Robot Program Synthesis Through Environmental Context](https://mlsanity.com/paper/2312.08250/) (2023)

## Built on by (most cited first, 1 of 1)

- [Lost in Tracking Translation: A Comprehensive Analysis of Visual SLAM in Human-Centered XR and IoT Ecosystems](https://mlsanity.com/paper/2411.07146/) (2024)

## Abstract (arXiv)

Autonomous robots, autonomous vehicles, and humans wearing mixed-reality headsets require accurate and reliable tracking services for safety-critical applications in dynamically changing real-world environments. However, the existing tracking approaches, such as Simultaneous Localization and Mapping (SLAM), do not adapt well to environmental changes and boundary conditions despite extensive manual tuning. On the other hand, while deep learning-based approaches can better adapt to environmental changes, they typically demand substantial data for training and often lack flexibility in adapting to new domains. To solve this problem, we propose leveraging the neurosymbolic program synthesis approach to construct adaptable SLAM pipelines that integrate the domain knowledge from traditional SLAM approaches while leveraging data to learn complex relationships. While the approach can synthesize end-to-end SLAM pipelines, we focus on synthesizing the feature extraction module. We first devise a domain-specific language (DSL) that can encapsulate domain knowledge on the important attributes for feature extraction and the real-world performance of various feature extractors. Our neurosymbolic architecture then undertakes adaptive feature extraction, optimizing parameters via learning while employing symbolic reasoning to select the most suitable feature extractor. Our evaluations demonstrate that our approach, neurosymbolic Feature EXtraction (nFEX), yields higher-quality features. It also reduces the pose error observed for the state-of-the-art baseline feature extractors ORB and SIFT by up to 90% and up to 66%, respectively, thereby enhancing the system's efficiency and adaptability to novel environments.