# Negative Frames Matter in Egocentric Visual Query 2D Localization

arXiv: https://arxiv.org/abs/2208.01949 · HTML: https://mlsanity.com/paper/2208.01949/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 2 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 2 of 2)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Finding Action Tubes](https://mlsanity.com/paper/1411.6031/) (2014)

## Built on by (most cited first, 14 of 14)

- [EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queries](https://mlsanity.com/paper/2212.06969/) (2022)
- [Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization](https://mlsanity.com/paper/2211.10528/) (2022)
- [Single-Stage Visual Query Localization in Egocentric Videos](https://mlsanity.com/paper/2306.09324/) (2023)
- [Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memory](https://mlsanity.com/paper/2411.16934/) (2024)
- [RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations](https://mlsanity.com/paper/2412.01826/) (2024)
- [PRVQL: Progressive Knowledge-guided Refinement for Robust Egocentric Visual Query Localization](https://mlsanity.com/paper/2502.07707/) (2025)
- [MINOTAUR: Multi-task Video Grounding From Multimodal Queries](https://mlsanity.com/paper/2302.08063/) (2023)
- [Estimating more camera poses for ego-centric videos is essential for VQ3D](https://mlsanity.com/paper/2211.10284/) (2022)
- [EgoCOL: Egocentric Camera pose estimation for Open-world 3D object Localization @Ego4D challenge 2023](https://mlsanity.com/paper/2306.16606/) (2023)
- [HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localization](https://mlsanity.com/paper/2509.00385/) (2025)
- [Towards Visual Query Localization in the 3D World](https://mlsanity.com/paper/2605.01498/) (2026)
- [Bayesian Decision Making to Localize Visual Queries in 2D](https://mlsanity.com/paper/2305.17611/) (2023)
- [EgoHieraLoc: A Cortically Inspired Hierarchical Segmentation-Guided Framework for Egocentric Visual Query Localization](https://mlsanity.com/paper/2608.09656/) (2026)
- [PGL-3D: Towards Progressive Geometric Learning for 3D Visual Query Localization](https://mlsanity.com/paper/2609.33558/) (2026)

## Abstract (arXiv)

The recently released Ego4D dataset and benchmark significantly scales and diversifies the first-person visual perception data. In Ego4D, the Visual Queries 2D Localization task aims to retrieve objects appeared in the past from the recording in the first-person view. This task requires a system to spatially and temporally localize the most recent appearance of a given object query, where query is registered by a single tight visual crop of the object in a different scene. Our study is based on the three-stage baseline introduced in the Episodic Memory benchmark. The baseline solves the problem by detection and tracking: detect the similar objects in all the frames, then run a tracker from the most confident detection result. In the VQ2D challenge, we identified two limitations of the current baseline. (1) The training configuration has redundant computation. Although the training set has millions of instances, most of them are repetitive and the number of unique object is only around 14.6k. The repeated gradient computation of the same object lead to an inefficient training; (2) The false positive rate is high on background frames. This is due to the distribution gap between training and evaluation. During training, the model is only able to see the clean, stable, and labeled frames, but the egocentric videos also have noisy, blurry, or unlabeled background frames. To this end, we developed a more efficient and effective solution. Concretely, we bring the training loop from ~15 days to less than 24 hours, and we achieve 0.17% spatial-temporal AP, which is 31% higher than the baseline. Our solution got the first ranking on the public leaderboard. Our code is publicly available at https://github.com/facebookresearch/vq2d_cvpr.