# NeuSE: Neural SE(3)-Equivariant Embedding for Long-Term Object-Based Simultaneous Localization and Mapping

arXiv: https://arxiv.org/abs/2303.07308 · HTML: https://mlsanity.com/paper/2303.07308/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 48 references that are in the map.
- First posted on arXiv 13 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [Vision-Only Robot Navigation in a Neural Radiance World](https://mlsanity.com/paper/2110.00168/) (2021)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects](https://mlsanity.com/paper/2110.14217/) (2021)
- [LENS: Localization enhanced by NeRF synthesis](https://mlsanity.com/paper/2110.06558/) (2021)
- [3D Neural Scene Representations for Visuomotor Control](https://mlsanity.com/paper/2107.04004/) (2021)
- [iSDF: Real-Time Neural Signed Distance Fields for Robot Perception](https://mlsanity.com/paper/2204.02296/) (2022)
- [Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations](https://mlsanity.com/paper/2104.01542/) (2021)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields](https://mlsanity.com/paper/2203.01913/) (2022)
- [Orbeez-SLAM: A Real-time Monocular Visual SLAM with ORB Features and NeRF-realized Mapping](https://mlsanity.com/paper/2209.13274/) (2022)
- [MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM](https://mlsanity.com/paper/1812.07976/) (2018)
- [NodeSLAM: Neural Object Descriptors for Multi-View Shape Reconstruction](https://mlsanity.com/paper/2004.04485/) (2020)
- [Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning](https://mlsanity.com/paper/2206.08321/) (2022)
- [SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields](https://mlsanity.com/paper/2211.09786/) (2022)
- [Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency](https://mlsanity.com/paper/2109.10165/) (2021)
- [SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representations](https://mlsanity.com/paper/1903.06482/) (2019)
- [DSP-SLAM: Object Oriented SLAM with Deep Shape Priors](https://mlsanity.com/paper/2108.09481/) (2021)

## Built on by (most cited first, 10 of 10)

- [Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation](https://mlsanity.com/paper/2309.02685/) (2023)
- [TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing](https://mlsanity.com/paper/2311.12602/) (2023)
- [SuperPrimitive: Scene Reconstruction at a Primitive Level](https://mlsanity.com/paper/2312.05889/) (2023)
- [Gaussian Mapping for Evolving Scenes](https://mlsanity.com/paper/2506.06909/) (2025)
- [LOSS-SLAM: Lightweight Open-Set Semantic Simultaneous Localization and Mapping](https://mlsanity.com/paper/2404.04377/) (2024)
- [BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding](https://mlsanity.com/paper/2412.02449/) (2024)
- [Perpetua: Multi-Hypothesis Persistence Modeling for Semi-Static Environments](https://mlsanity.com/paper/2507.18808/) (2025)
- [DEF-oriCORN: efficient 3D scene understanding for robust language-directed manipulation without demonstrations](https://mlsanity.com/paper/2407.21267/) (2024)
- [On Uncertainty Calibration for Equivariant Functions](https://mlsanity.com/paper/2510.21691/) (2025)
- [Visual General Intelligence: A White Paper](https://mlsanity.com/paper/2608.25924/) (2026)

## Abstract (arXiv)

We present NeuSE, a novel Neural SE(3)-Equivariant Embedding for objects, and illustrate how it supports object-based Simultaneous Localization and Mapping (SLAM) for consistent spatial understanding with long-term scene changes. NeuSE is a set of latent object embeddings created from partial object observations. It serves as a compact point cloud surrogate for complete object models, encoding the full shape, scale, and transform information about an object. In addition, the inferred latent code is both SE(3) and scale equivariant, enabling strong generalization to objects of both unseen sizes and different SE(3) poses. This makes NeuSE particularly effective in real-world scenarios where objects may vary in size or spatial configuration. With NeuSE, relative frame transforms can be directly derived from inferred latent codes. Our proposed SLAM paradigm, using NeuSE for object shape, size, and pose characterization, can operate independently or in conjunction with typical SLAM systems. It directly infers SE(3) camera pose constraints that are compatible with general SLAM pose graph optimization, while maintaining a lightweight, object-centric map that adapts to real-world changes. Our evaluation is conducted on synthetic and real-world sequences with changes in both controlled and uncontrolled settings, featuring multi-category objects of various shapes and sizes. Our approach demonstrates improved localization capability and change-aware mapping consistency when working either independently or as a complement to common SLAM pipelines. Additional results are presented in the supplementary video: https://youtu.be/fZ4_2P1fQBQ