# RoboHop: Segment-based Topological Map Representation for Open-World Visual Navigation

arXiv: https://arxiv.org/abs/2405.05792 · HTML: https://mlsanity.com/paper/2405.05792/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 72 references that are in the map.
- First posted on arXiv 9 May 2024.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning](https://mlsanity.com/paper/2307.06135/) (2023)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Neural Topological SLAM for Visual Navigation](https://mlsanity.com/paper/2005.12256/) (2020)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Learning Exploration Policies for Navigation](https://mlsanity.com/paper/1903.01959/) (2019)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav](https://mlsanity.com/paper/2301.07302/) (2023)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [3-D Scene Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agents](https://mlsanity.com/paper/1908.04929/) (2019)
- [SeqNet: Learning Descriptors for Sequence-based Hierarchical Place Recognition](https://mlsanity.com/paper/2102.11603/) (2021)
- [How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers](https://mlsanity.com/paper/2305.16925/) (2023)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)

## Built on by (most cited first, 25 of 25)

- [3D-Mem: 3D Scene Memory for Embodied Exploration and Reasoning](https://mlsanity.com/paper/2411.17735/) (2024)
- [Revisit Anything: Visual Place Recognition via Image Segment Retrieval](https://mlsanity.com/paper/2409.18049/) (2024)
- [Describe Anything Anywhere At Any Moment](https://mlsanity.com/paper/2512.00565/) (2025)
- [SPINE: Online Semantic Planning for Missions with Incomplete Natural Language Specifications in Unstructured Environments](https://mlsanity.com/paper/2410.03035/) (2024)
- [SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing](https://mlsanity.com/paper/2506.05414/) (2025)
- [Duoduo CLIP: Efficient 3D Understanding with Multi-View Images](https://mlsanity.com/paper/2406.11579/) (2024)
- [Air-Ground Collaboration for Language-Specified Missions in Unknown Environments](https://mlsanity.com/paper/2505.09108/) (2025)
- [Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments](https://mlsanity.com/paper/2410.20666/) (2024)
- [Multiview Scene Graph](https://mlsanity.com/paper/2410.11187/) (2024)
- [ForesightNav: Learning Scene Imagination for Efficient Exploration](https://mlsanity.com/paper/2504.16062/) (2025)
- [TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goals](https://mlsanity.com/paper/2509.08699/) (2025)
- [ObjectReact: Learning Object-Relative Control for Visual Navigation](https://mlsanity.com/paper/2509.09594/) (2025)
- [SegMASt3R: Geometry Grounded Segment Matching](https://mlsanity.com/paper/2510.05051/) (2025)
- [VLAD-BuFF: Burst-aware Fast Feature Aggregation for Visual Place Recognition](https://mlsanity.com/paper/2409.19293/) (2024)
- [Deploying Foundation Model-Enabled Air and Ground Robots in the Field: Challenges and Opportunities](https://mlsanity.com/paper/2505.09477/) (2025)
- [Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering](https://mlsanity.com/paper/2510.01483/) (2025)
- [SUM-AgriVLN: Spatial Understanding Memory for Agricultural Vision-and-Language Navigation](https://mlsanity.com/paper/2510.14357/) (2025)
- [FOM-Nav: Frontier-Object Maps for Object Goal Navigation](https://mlsanity.com/paper/2512.01009/) (2025)
- [MASt3R-Nav: WayPixel Navigation in Relative 3D Maps](https://mlsanity.com/paper/2605.24111/) (2026)
- [EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation](https://mlsanity.com/paper/2606.03509/) (2026)
- [4D-CS: Exploiting Cluster Prior for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2501.02937/) (2025)
- [PlatonicNav: Unveiling Semantic Correspondence in Navigation with Platonic Topological Maps](https://mlsanity.com/paper/2606.01788/) (2026)
- [PixelLoop: Shortcut Topological Navigation with Pixel-Level Loops](https://mlsanity.com/paper/2607.12811/) (2026)
- [SceneBind: Binding What and Where Across Vision, Audio and Language](https://mlsanity.com/paper/2607.15265/) (2026)
- [MuViSeg: Multi-View Segment Correspondences from Dense Geometry Priors](https://mlsanity.com/paper/2607.17938/) (2026)

## Abstract (arXiv)

Mapping is crucial for spatial reasoning, planning and robot navigation. Existing approaches range from metric, which require precise geometry-based optimization, to purely topological, where image-as-node based graphs lack explicit object-level reasoning and interconnectivity. In this paper, we propose a novel topological representation of an environment based on "image segments", which are semantically meaningful and open-vocabulary queryable, conferring several advantages over previous works based on pixel-level features. Unlike 3D scene graphs, we create a purely topological graph with segments as nodes, where edges are formed by a) associating segment-level descriptors between pairs of consecutive images and b) connecting neighboring segments within an image using their pixel centroids. This unveils a "continuous sense of a place", defined by inter-image persistence of segments along with their intra-image neighbours. It further enables us to represent and update segment-level descriptors through neighborhood aggregation using graph convolution layers, which improves robot localization based on segment-level retrieval. Using real-world data, we show how our proposed map representation can be used to i) generate navigation plans in the form of "hops over segments" and ii) search for target objects using natural language queries describing spatial relations of objects. Furthermore, we quantitatively analyze data association at the segment level, which underpins inter-image connectivity during mapping and segment-level localization when revisiting the same place. Finally, we show preliminary trials on segment-level `hopping' based zero-shot real-world navigation. Project page with supplementary details: oravus.github.io/RoboHop/