# ObjectReact: Learning Object-Relative Control for Visual Navigation

arXiv: https://arxiv.org/abs/2509.09594 · HTML: https://mlsanity.com/paper/2509.09594/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 91 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 30 of 60)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [RoMa: Robust Dense Feature Matching](https://mlsanity.com/paper/2305.15404/) (2023)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation](https://mlsanity.com/paper/2403.17846/) (2024)
- [Neural Topological SLAM for Visual Navigation](https://mlsanity.com/paper/2005.12256/) (2020)
- [GNM: A General Navigation Model to Drive Any Robot](https://mlsanity.com/paper/2210.03370/) (2022)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [Rapid Exploration for Open-World Navigation with Latent Goal Models](https://mlsanity.com/paper/2104.05859/) (2021)
- [Zero-Shot Visual Imitation](https://mlsanity.com/paper/1804.08606/) (2018)
- [SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation](https://mlsanity.com/paper/2410.08189/) (2024)
- [ViNG: Learning Open-World Navigation with Visual Goals](https://mlsanity.com/paper/2012.09812/) (2020)
- [Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill](https://mlsanity.com/paper/2309.10309/) (2023)
- [Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation](https://mlsanity.com/paper/2303.03480/) (2023)
- [Bird's-Eye-View Scene Graph for Vision-Language Navigation](https://mlsanity.com/paper/2308.04758/) (2023)

## Built on by (most cited first, 5 of 5)

- [SegMASt3R: Geometry Grounded Segment Matching](https://mlsanity.com/paper/2510.05051/) (2025)
- [MASt3R-Nav: WayPixel Navigation in Relative 3D Maps](https://mlsanity.com/paper/2605.24111/) (2026)
- [UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following](https://mlsanity.com/paper/2607.13621/) (2026)
- [PlatonicNav: Unveiling Semantic Correspondence in Navigation with Platonic Topological Maps](https://mlsanity.com/paper/2606.01788/) (2026)
- [PixelLoop: Shortcut Topological Navigation with Pixel-Level Loops](https://mlsanity.com/paper/2607.12811/) (2026)

## Abstract (arXiv)

Visual navigation using only a single camera and a topological map has recently become an appealing alternative to methods that require additional sensors and 3D maps. This is typically achieved through an "image-relative" approach to estimating control from a given pair of current observation and subgoal image. However, image-level representations of the world have limitations because images are strictly tied to the agent's pose and embodiment. In contrast, objects, being a property of the map, offer an embodiment- and trajectory-invariant world representation. In this work, we present a new paradigm of learning "object-relative" control that exhibits several desirable characteristics: a) new routes can be traversed without strictly requiring to imitate prior experience, b) the control prediction problem can be decoupled from solving the image matching problem, and c) high invariance can be achieved in cross-embodiment deployment for variations across both training-testing and mapping-execution settings. We propose a topometric map representation in the form of a "relative" 3D scene graph, which is used to obtain more informative object-level global path planning costs. We train a local controller, dubbed "ObjectReact", conditioned directly on a high-level "WayObject Costmap" representation that eliminates the need for an explicit RGB input. We demonstrate the advantages of learning object-relative control over its image-relative counterpart across sensor height variations and multiple navigation tasks that challenge the underlying spatial understanding capability, e.g., navigating a map trajectory in the reverse direction. We further show that our sim-only policy is able to generalize well to real-world indoor environments. Code and supplementary material are accessible via project page: https://object-react.github.io/