# Towards Optimal Correlational Object Search

arXiv: https://arxiv.org/abs/2110.09991 · HTML: https://mlsanity.com/paper/2110.09991/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 48 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects](https://mlsanity.com/paper/2006.13171/) (2020)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning](https://mlsanity.com/paper/1812.00971/) (2018)
- [The Marathon 2: A Navigation System](https://mlsanity.com/paper/2003.00368/) (2020)
- [Partially Observable Markov Decision Processes (POMDPs) and Robotics](https://mlsanity.com/paper/2107.07599/) (2021)
- [Visual Navigation with Spatial Attention](https://mlsanity.com/paper/2104.09807/) (2021)
- [Object Finding in Cluttered Scenes Using Interactive Perception](https://mlsanity.com/paper/1911.07482/) (2019)
- [Semantic Linking Maps for Active Visual Object Search](https://mlsanity.com/paper/2006.10807/) (2020)
- [MAGIC: Learning Macro-Actions for Online POMDP Planning](https://mlsanity.com/paper/2011.03813/) (2020)
- [ROS Navigation Tuning Guide](https://mlsanity.com/paper/1706.09068/) (2017)
- [Multi-Resolution POMDP Planning for Multi-Object Search in 3D](https://mlsanity.com/paper/2005.02878/) (2020)
- [Spatial Language Understanding for Object Search in Partially Observed City-scale Environments](https://mlsanity.com/paper/2012.02705/) (2020)
- [From Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping](https://mlsanity.com/paper/1812.11866/) (2018)
- [pomdp_py: A Framework to Build and Solve POMDP Problems](https://mlsanity.com/paper/2004.10099/) (2020)
- [RoboMem: Giving Long Term Memory to Robots](https://mlsanity.com/paper/2003.10553/) (2020)

## Built on by (most cited first, 14 of 14)

- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Modeling Dynamic Environments with Scene Graph Memory](https://mlsanity.com/paper/2305.17537/) (2023)
- [Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation](https://mlsanity.com/paper/2307.06125/) (2023)
- [A System for Generalized 3D Multi-Object Search](https://mlsanity.com/paper/2303.03178/) (2023)
- [Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigation](https://mlsanity.com/paper/2302.01520/) (2023)
- [Zero-shot Active Visual Search (ZAVIS): Intelligent Object Search for Robotic Assistants](https://mlsanity.com/paper/2209.08803/) (2022)
- [IPPON: Common Sense Guided Informative Path Planning for Object Goal Navigation](https://mlsanity.com/paper/2410.19697/) (2024)
- [CuriousBot: Interactive Mobile Exploration via Actionable 3D Relational Object Graph](https://mlsanity.com/paper/2501.13338/) (2025)
- [Large-Language-Model-Guided State Estimation for Partially Observable Task and Motion Planning](https://mlsanity.com/paper/2603.03704/) (2026)
- [Hierarchical Object-Oriented POMDP Planning for Object Rearrangement](https://mlsanity.com/paper/2412.01348/) (2024)
- [Align While Search: Belief-Guided Exploratory Inference for World-Grounded Embodied Agents](https://mlsanity.com/paper/2512.24461/) (2025)
- [Embodied Active Learning under Limited Annotation and Navigation Budget for Object Detection](https://mlsanity.com/paper/2607.15974/) (2026)
- [Beyond the Remembered World: Predictive 4D Belief for Persistent Navigation in Evolving Worlds](https://mlsanity.com/paper/2609.39166/) (2026)

## Abstract (arXiv)

In realistic applications of object search, robots will need to locate target objects in complex environments while coping with unreliable sensors, especially for small or hard-to-detect objects. In such settings, correlational information can be valuable for planning efficiently. Previous approaches that consider correlational information typically resort to ad-hoc, greedy search strategies. We introduce the Correlational Object Search POMDP (COS-POMDP), which models correlations while preserving optimal solutions with a reduced state space. We propose a hierarchical planning algorithm to scale up COS-POMDPs for practical domains. Our evaluation, conducted with the AI2-THOR household simulator and the YOLOv5 object detector, shows that our method finds objects more successfully and efficiently compared to baselines,particularly for hard-to-detect objects such as srub brush and remote control.