# EgoHieraLoc: A Cortically Inspired Hierarchical Segmentation-Guided Framework for Egocentric Visual Query Localization

arXiv: https://arxiv.org/abs/2608.09656 · HTML: https://mlsanity.com/paper/2608.09656/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 54 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Builds on (most cited first, 8 of 8)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [EgoLifter: Open-world 3D Segmentation for Egocentric Perception](https://mlsanity.com/paper/2403.18118/) (2024)
- [EgoGaussian: Dynamic Scene Understanding from Egocentric Video with 3D Gaussian Splatting](https://mlsanity.com/paper/2406.19811/) (2024)
- [Negative Frames Matter in Egocentric Visual Query 2D Localization](https://mlsanity.com/paper/2208.01949/) (2022)
- [PRVQL: Progressive Knowledge-guided Refinement for Robust Egocentric Visual Query Localization](https://mlsanity.com/paper/2502.07707/) (2025)
- [Estimating more camera poses for ego-centric videos is essential for VQ3D](https://mlsanity.com/paper/2211.10284/) (2022)
- [EgoCOL: Egocentric Camera pose estimation for Open-world 3D object Localization @Ego4D challenge 2023](https://mlsanity.com/paper/2306.16606/) (2023)

## Abstract (arXiv)

Visual query localization (VQL) aims to retrieve and re-localize a queried object in egocentric videos, yet remains challenging when object boundaries are ambiguous and global context cannot effectively guide fine-grained localization. Human vision handles such ambiguity through a hierarchical process: it rapidly screens foreground candidates, selectively attends to the target despite distractors, refines perception via feedback between global context and local detail, and, when a single view is unreliable, integrates evidence across viewpoints according to its credibility. Inspired by these competencies, we propose EgoHieraLoc, a unified framework for VQL-2D and VQL-3D. A Discriminative Parsing Module first extracts foreground-aware query representations using segmentation priors; a Query-Aware Module then performs robust target localization through discriminative correlation filtering with deformable modeling; and a Regional Adaptation Module feeds multi-scale context back into local regions to recover precise object boundaries. To extend this perceptual hierarchy to 3D localization, we introduce Geometric-Semantic Joint Confidence (GSJC), which multiplicatively couples segmentation confidence with local depth consistency, multi-view back-projection consistency, and triangulation-baseline quality, so that a viewpoint contributes to the 3D estimate only when it is credible both semantically and geometrically. Extensive experiments demonstrate state-of-the-art performance on both VQL-2D and -3D benchmarks.