# ObjectNLQ @ Ego4D Episodic Memory Challenge 2024

arXiv: https://arxiv.org/abs/2406.15778 · HTML: https://mlsanity.com/paper/2406.15778/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 22 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [DETRs with Collaborative Hybrid Assignments Training](https://mlsanity.com/paper/2211.12860/) (2022)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges](https://mlsanity.com/paper/2211.09529/) (2022)
- [NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory](https://mlsanity.com/paper/2301.00746/) (2023)
- [CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding](https://mlsanity.com/paper/2209.10918/) (2022)
- [GroundNLQ @ Ego4D Natural Language Queries Challenge 2023](https://mlsanity.com/paper/2306.15255/) (2023)
- [ReLER@ZJU-Alibaba Submission to the Ego4D Natural Language Queries Challenge 2022](https://mlsanity.com/paper/2207.00383/) (2022)
- [Action Sensitivity Learning for the Ego4D Episodic Memory Challenge 2023](https://mlsanity.com/paper/2306.09172/) (2023)
- [Egocentric Video-Language Pretraining @ Ego4D Challenge 2022](https://mlsanity.com/paper/2207.01622/) (2022)
- [A Simple Transformer-Based Model for Ego4D Natural Language Queries Challenge](https://mlsanity.com/paper/2211.08704/) (2022)

## Built on by (most cited first, 10 of 10)

- [Spatial Understanding from Videos: Structured Prompts Meet Simulation Data](https://mlsanity.com/paper/2506.03642/) (2025)
- [Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding](https://mlsanity.com/paper/2503.09143/) (2025)
- [Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memory](https://mlsanity.com/paper/2411.16934/) (2024)
- [Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025](https://mlsanity.com/paper/2506.02550/) (2025)
- [HCQA-1.5 @ Ego4D EgoSchema Challenge 2025](https://mlsanity.com/paper/2505.20644/) (2025)
- [OSGNet @ Ego4D Episodic Memory Challenge 2025](https://mlsanity.com/paper/2506.03710/) (2025)
- [MARS: Technical Report for the CASTLE Challenge at EgoVis 2026](https://mlsanity.com/paper/2605.18176/) (2026)
- [GazeNLQ @ Ego4D Natural Language Queries Challenge 2025](https://mlsanity.com/paper/2506.05782/) (2025)
- [OSGNet with MLLM Reranking @ Ego4D Episodic Memory Challenge 2026](https://mlsanity.com/paper/2605.20818/) (2026)
- [Hand Trajectory Fusion for Egocentric Natural Language Query Grounding](https://mlsanity.com/paper/2606.02962/) (2026)

## Abstract (arXiv)

In this report, we present our approach for the Natural Language Query track and Goal Step track of the Ego4D Episodic Memory Benchmark at CVPR 2024. Both challenges require the localization of actions within long video sequences using textual queries. To enhance localization accuracy, our method not only processes the temporal information of videos but also identifies fine-grained objects spatially within the frames. To this end, we introduce a novel approach, termed ObjectNLQ, which incorporates an object branch to augment the video representation with detailed object information, thereby improving grounding efficiency. ObjectNLQ achieves a mean R@1 of 23.15, ranking 2nd in the Natural Language Queries Challenge, and gains 33.00 in terms of the metric R@1, IoU=0.3, ranking 3rd in the Goal Step Challenge. Our code will be released at https://github.com/Yisen-Feng/ObjectNLQ.