# Maintaining a Reliable World Model using Action-aware Perceptual Anchoring

arXiv: https://arxiv.org/abs/2107.03038 · HTML: https://mlsanity.com/paper/2107.03038/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 4 of 4)

- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning](https://mlsanity.com/paper/1910.04744/) (2019)
- [Learning Object Permanence from Video](https://mlsanity.com/paper/2003.10469/) (2020)
- [Semantic Relational Object Tracking](https://mlsanity.com/paper/1902.09937/) (2019)

## Built on by (most cited first, 1 of 1)

- [Improving Object Permanence using Agent Actions and Reasoning](https://mlsanity.com/paper/2110.00238/) (2021)

## Abstract (arXiv)

Reliable perception is essential for robots that interact with the world. But sensors alone are often insufficient to provide this capability, and they are prone to errors due to various conditions in the environment. Furthermore, there is a need for robots to maintain a model of its surroundings even when objects go out of view and are no longer visible. This requires anchoring perceptual information onto symbols that represent the objects in the environment. In this paper, we present a model for action-aware perceptual anchoring that enables robots to track objects in a persistent manner. Our rule-based approach considers inductive biases to perform high-level reasoning over the results from low-level object detection, and it improves the robot's perceptual capability for complex tasks. We evaluate our model against existing baseline models for object permanence and show that it outperforms these on a snitch localisation task using a dataset of 1,371 videos. We also integrate our action-aware perceptual anchoring in the context of a cognitive architecture and demonstrate its benefits in a realistic gearbox assembly task on a Universal Robot.