# EMPNet: Neural Localisation and Mapping Using Embedded Memory Points

arXiv: https://arxiv.org/abs/1907.13268 · HTML: https://mlsanity.com/paper/1907.13268/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2019.

## Built on by (most cited first, 9 of 9)

- [SOON: Scenario Oriented Object Navigation with Graph-based Exploration](https://mlsanity.com/paper/2103.17138/) (2021)
- [Memory-Augmented Reinforcement Learning for Image-Goal Navigation](https://mlsanity.com/paper/2101.05181/) (2021)
- [FGPrompt: Fine-grained Goal Prompting for Image-goal Navigation](https://mlsanity.com/paper/2310.07473/) (2023)
- [Overcoming Obstructions via Bandwidth-Limited Multi-Agent Spatial Handshaking](https://mlsanity.com/paper/2107.00771/) (2021)
- [Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation](https://mlsanity.com/paper/2206.01916/) (2022)
- [Image-Goal Navigation Using Refined Feature Guidance and Scene Graph Enhancement](https://mlsanity.com/paper/2503.10986/) (2025)
- [Deep Weakly Supervised Positioning](https://mlsanity.com/paper/2104.04866/) (2021)
- [Enhancing Multi-Robot Perception via Learned Data Association](https://mlsanity.com/paper/2107.00769/) (2021)
- [Semantic Environment Atlas for Object-Goal Navigation](https://mlsanity.com/paper/2410.09081/) (2024)

## Abstract (arXiv)

Continuously estimating an agent's state space and a representation of its surroundings has proven vital towards full autonomy. A shared common ground among systems which successfully achieve this feat is the integration of previously encountered observations into the current state being estimated. This necessitates the use of a memory module for incorporating previously visited states whilst simultaneously offering an internal representation of the observed environment. In this work we develop a memory module which contains rigidly aligned point-embeddings that represent a coherent scene structure acquired from an RGB-D sequence of observations. The point-embeddings are extracted using modern convolutional neural network architectures, and alignment is performed by computing a dense correspondence matrix between a new observation and the current embeddings residing in the memory module. The whole framework is end-to-end trainable, resulting in a recurrent joint optimisation of the point-embeddings contained in the memory. This process amplifies the shared information across states, providing increased robustness and accuracy. We show significant improvement of our method across a set of experiments performed on the synthetic VIZDoom environment and a real world Active Vision Dataset.