# MEM: Multi-Modal Elevation Mapping for Robotics and Learning

arXiv: https://arxiv.org/abs/2309.16818 · HTML: https://mlsanity.com/paper/2309.16818/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 12 of 12)

- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning](https://mlsanity.com/paper/1611.03631/) (2016)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [Fast Traversability Estimation for Wild Visual Navigation](https://mlsanity.com/paper/2305.08510/) (2023)
- [Elevation Mapping for Locomotion and Navigation using GPU](https://mlsanity.com/paper/2204.12876/) (2022)
- [End-to-end Lane Detection through Differentiable Least-Squares Fitting](https://mlsanity.com/paper/1902.00293/) (2019)
- [Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping](https://mlsanity.com/paper/1909.04631/) (2019)
- [Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments](https://mlsanity.com/paper/2203.15854/) (2022)
- [These Maps Are Made For Walking: Real-Time Terrain Property Estimation for Mobile Robots](https://mlsanity.com/paper/2205.12925/) (2022)
- [Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference](https://mlsanity.com/paper/2106.14986/) (2021)

## Built on by (most cited first, 11 of 11)

- [RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving](https://mlsanity.com/paper/2402.19341/) (2024)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision](https://mlsanity.com/paper/2404.07110/) (2024)
- [PIETRA: Physics-Informed Evidential Learning for Traversing Out-of-Distribution Terrain](https://mlsanity.com/paper/2409.03005/) (2024)
- [VAMOS: A Hierarchical Vision-Language-Action Model for Capability-Modulated and Steerable Navigation](https://mlsanity.com/paper/2510.20818/) (2025)
- [PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation](https://mlsanity.com/paper/2511.18570/) (2025)
- [Learning Semantic Traversability with Egocentric Video and Automated Annotation Strategy](https://mlsanity.com/paper/2406.02989/) (2024)
- [TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation](https://mlsanity.com/paper/2404.15256/) (2024)
- [Dynamics Modeling using Visual Terrain Features for High-Speed Autonomous Off-Road Driving](https://mlsanity.com/paper/2412.00581/) (2024)
- [Splatblox: Traversability-Aware Gaussian Splatting for Outdoor Robot Navigation](https://mlsanity.com/paper/2511.18525/) (2025)
- [Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation](https://mlsanity.com/paper/2609.17141/) (2026)

## Abstract (arXiv)

Elevation maps are commonly used to represent the environment of mobile robots and are instrumental for locomotion and navigation tasks. However, pure geometric information is insufficient for many field applications that require appearance or semantic information, which limits their applicability to other platforms or domains. In this work, we extend a 2.5D robot-centric elevation mapping framework by fusing multi-modal information from multiple sources into a popular map representation. The framework allows inputting data contained in point clouds or images in a unified manner. To manage the different nature of the data, we also present a set of fusion algorithms that can be selected based on the information type and user requirements. Our system is designed to run on the GPU, making it real-time capable for various robotic and learning tasks. We demonstrate the capabilities of our framework by deploying it on multiple robots with varying sensor configurations and showcasing a range of applications that utilize multi-modal layers, including line detection, human detection, and colorization.