# Uni-Fusion: Universal Continuous Mapping

arXiv: https://arxiv.org/abs/2303.12678 · HTML: https://mlsanity.com/paper/2303.12678/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 62 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 30 of 33)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory](https://mlsanity.com/paper/2210.05663/) (2022)
- [Orthogonal Random Features](https://mlsanity.com/paper/1610.09072/) (2016)
- [DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors](https://mlsanity.com/paper/2012.05551/) (2020)
- [Multi-style Generative Network for Real-time Transfer](https://mlsanity.com/paper/1703.06953/) (2017)
- [Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2108.06230/) (2021)
- [Revisiting Image Pyramid Structure for High Resolution Salient Object Detection](https://mlsanity.com/paper/2209.09475/) (2022)
- [Zero-shot Learning of 3D Point Cloud Objects](https://mlsanity.com/paper/1902.10272/) (2019)
- [BNV-Fusion: Dense 3D Reconstruction using Bi-level Neural Volume Fusion](https://mlsanity.com/paper/2204.01139/) (2022)
- [Quadrature-based features for kernel approximation](https://mlsanity.com/paper/1802.03832/) (2018)
- [Probabilistic Volumetric Fusion for Dense Monocular SLAM](https://mlsanity.com/paper/2210.01276/) (2022)
- [NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping](https://mlsanity.com/paper/2110.09415/) (2021)
- [Gaussian Process Autonomous Mapping and Exploration for Range Sensing Mobile Robots](https://mlsanity.com/paper/1605.00335/) (2016)

## Built on by (most cited first, 5 of 5)

- [Neural Fields in Robotics: A Survey](https://mlsanity.com/paper/2410.20220/) (2024)
- [NEDS-SLAM: A Neural Explicit Dense Semantic SLAM Framework using 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11679/) (2024)
- [Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM](https://mlsanity.com/paper/2403.19473/) (2024)
- [OpenGS-Fusion: Open-Vocabulary Dense Mapping with Hybrid 3D Gaussian Splatting for Refined Object-Level Understanding](https://mlsanity.com/paper/2508.01150/) (2025)
- [SceneFactory: A Workflow-centric and Unified Framework for Incremental Scene Modeling](https://mlsanity.com/paper/2405.07847/) (2024)

## Abstract (arXiv)

We present Uni-Fusion, a universal continuous mapping framework for surfaces, surface properties (color, infrared, etc.) and more (latent features in CLIP embedding space, etc.). We propose the first universal implicit encoding model that supports encoding of both geometry and different types of properties (RGB, infrared, features, etc.) without requiring any training. Based on this, our framework divides the point cloud into regular grid voxels and generates a latent feature in each voxel to form a Latent Implicit Map (LIM) for geometries and arbitrary properties. Then, by fusing a local LIM frame-wisely into a global LIM, an incremental reconstruction is achieved. Encoded with corresponding types of data, our Latent Implicit Map is capable of generating continuous surfaces, surface property fields, surface feature fields, and all other possible options. To demonstrate the capabilities of our model, we implement three applications: (1) incremental reconstruction for surfaces and color (2) 2D-to-3D transfer of fabricated properties (3) open-vocabulary scene understanding by creating a text CLIP feature field on surfaces. We evaluate Uni-Fusion by comparing it in corresponding applications, from which Uni-Fusion shows high-flexibility in various applications while performing best or being competitive. The project page of Uni-Fusion is available at https://jarrome.github.io/Uni-Fusion/ .