# FastMap: Real-Time Semantic Map Completion via Bitwise Masked Modeling

arXiv: https://arxiv.org/abs/2506.07350 · HTML: https://mlsanity.com/paper/2506.07350/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis](https://mlsanity.com/paper/2211.09117/) (2022)
- [Neural Topological SLAM for Visual Navigation](https://mlsanity.com/paper/2005.12256/) (2020)
- [Simple but Effective: CLIP Embeddings for Embodied AI](https://mlsanity.com/paper/2111.09888/) (2021)
- [PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning](https://mlsanity.com/paper/2201.10029/) (2022)
- [MMM: Generative Masked Motion Model](https://mlsanity.com/paper/2312.03596/) (2023)
- [MaskBit: Embedding-free Image Generation via Bit Tokens](https://mlsanity.com/paper/2409.16211/) (2024)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification](https://mlsanity.com/paper/2212.00338/) (2022)
- [SSCNav: Confidence-Aware Semantic Scene Completion for Visual Semantic Navigation](https://mlsanity.com/paper/2012.04512/) (2020)
- [PEANUT: Predicting and Navigating to Unseen Targets](https://mlsanity.com/paper/2212.02497/) (2022)
- [ForesightNav: Learning Scene Imagination for Efficient Exploration](https://mlsanity.com/paper/2504.16062/) (2025)
- [Don't Look into the Dark: Latent Codes for Pluralistic Image Inpainting](https://mlsanity.com/paper/2403.18186/) (2024)

## Abstract (arXiv)

Semantic map completion, which predicts the layout of unobserved regions from partial observations, is a critical capability for indoor robot navigation. Existing approaches either rely on high-dimensional discrete codebooks that inflate memory, or on iterative diffusion sampling that is too slow for real-time use. We present FastMap, a lightweight two-stage framework for completing top-down categorical semantic maps. First, a lookup-free BitVAE exploits the inherently binary (one-hot) structure of semantic maps to compress each map patch into compact bitwise tokens, yielding a 0.41GB model that is 3.7 times smaller than the prior masked-modeling baseline. Second, a Masked AutoEncoder (MAE)-style transformer reconstructs missing tokens in a single forward pass at 0.011s/map. To support object goal navigation, we additionally introduce an object-aware masking strategy that masks the target category during training and conditions generation on a learnable category embedding, without adding inference cost. On the Gibson benchmark, FastMap achieves 34.10% mIoU and 45.84% semantic Success Rate (sSR), more than doubling the previous best (21.88%), and reaches 83.8% Success Rate on downstream ObjectNav, the highest among the compared navigation methods.