# An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 2022

arXiv: https://arxiv.org/abs/2211.08776 · HTML: https://mlsanity.com/paper/2211.08776/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 6 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 5 of 5)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [Egocentric Video-Language Pretraining](https://mlsanity.com/paper/2206.01670/) (2022)
- [CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding](https://mlsanity.com/paper/2209.10918/) (2022)
- [ReLER@ZJU-Alibaba Submission to the Ego4D Natural Language Queries Challenge 2022](https://mlsanity.com/paper/2207.00383/) (2022)

## Built on by (most cited first, 4 of 4)

- [NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory](https://mlsanity.com/paper/2301.00746/) (2023)
- [GroundNLQ @ Ego4D Natural Language Queries Challenge 2023](https://mlsanity.com/paper/2306.15255/) (2023)
- [SpotEM: Efficient Video Search for Episodic Memory](https://mlsanity.com/paper/2306.15850/) (2023)
- [CARLOR @ Ego4D Step Grounding Challenge: Bayesian temporal-order priors for test time refinement](https://mlsanity.com/paper/2406.09575/) (2024)

## Abstract (arXiv)

This technical report describes the CONE approach for Ego4D Natural Language Queries (NLQ) Challenge in ECCV 2022. We leverage our model CONE, an efficient window-centric COarse-to-fiNE alignment framework. Specifically, CONE dynamically slices the long video into candidate windows via a sliding window approach. Centering at windows, CONE (1) learns the inter-window (coarse-grained) semantic variance through contrastive learning and speeds up inference by pre-filtering the candidate windows relevant to the NL query, and (2) conducts intra-window (fine-grained) candidate moments ranking utilizing the powerful multi-modal alignment ability of the contrastive vision-text pre-trained model EgoVLP. On the blind test set, CONE achieves 15.26 and 9.24 for R1@IoU=0.3 and R1@IoU=0.5, respectively.