# ReLER@ZJU-Alibaba Submission to the Ego4D Natural Language Queries Challenge 2022

arXiv: https://arxiv.org/abs/2207.00383 · HTML: https://mlsanity.com/paper/2207.00383/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 13 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [Omnivore: A Single Model for Many Visual Modalities](https://mlsanity.com/paper/2201.08377/) (2022)
- [T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval](https://mlsanity.com/paper/2104.10054/) (2021)
- [CenterCLIP: Token Clustering for Efficient Text-Video Retrieval](https://mlsanity.com/paper/2205.00823/) (2022)
- [QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries](https://mlsanity.com/paper/2107.09609/) (2021)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/2012.02646/) (2020)

## Built on by (most cited first, 19 of 19)

- [Grounded Question-Answering in Long Egocentric Videos](https://mlsanity.com/paper/2312.06505/) (2023)
- [LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos](https://mlsanity.com/paper/2312.05269/) (2023)
- [NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory](https://mlsanity.com/paper/2301.00746/) (2023)
- [Action Sensitivity Learning for Temporal Action Localization](https://mlsanity.com/paper/2305.15701/) (2023)
- [EgoEnv: Human-centric environment representations from egocentric video](https://mlsanity.com/paper/2207.11365/) (2022)
- [GroundNLQ @ Ego4D Natural Language Queries Challenge 2023](https://mlsanity.com/paper/2306.15255/) (2023)
- [ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos](https://mlsanity.com/paper/2411.14901/) (2024)
- [SpotEM: Efficient Video Search for Episodic Memory](https://mlsanity.com/paper/2306.15850/) (2023)
- [RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos](https://mlsanity.com/paper/2312.06729/) (2023)
- [Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memory](https://mlsanity.com/paper/2411.16934/) (2024)
- [Object-Shot Enhanced Grounding Network for Egocentric Video](https://mlsanity.com/paper/2505.04270/) (2025)
- [Boundary-Denoising for Video Activity Localization](https://mlsanity.com/paper/2304.02934/) (2023)
- [ObjectNLQ @ Ego4D Episodic Memory Challenge 2024](https://mlsanity.com/paper/2406.15778/) (2024)
- [Action Sensitivity Learning for the Ego4D Episodic Memory Challenge 2023](https://mlsanity.com/paper/2306.09172/) (2023)
- [A Simple Transformer-Based Model for Ego4D Natural Language Queries Challenge](https://mlsanity.com/paper/2211.08704/) (2022)
- [MINOTAUR: Multi-task Video Grounding From Multimodal Queries](https://mlsanity.com/paper/2302.08063/) (2023)
- [An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 2022](https://mlsanity.com/paper/2211.08776/) (2022)
- [MomentMix Augmentation with Length-Aware DETR for Temporally Robust Moment Retrieval](https://mlsanity.com/paper/2412.20816/) (2024)
- [GazeNLQ @ Ego4D Natural Language Queries Challenge 2025](https://mlsanity.com/paper/2506.05782/) (2025)

## Abstract (arXiv)

In this report, we present the ReLER@ZJU-Alibaba submission to the Ego4D Natural Language Queries (NLQ) Challenge in CVPR 2022. Given a video clip and a text query, the goal of this challenge is to locate a temporal moment of the video clip where the answer to the query can be obtained. To tackle this task, we propose a multi-scale cross-modal transformer and a video frame-level contrastive loss to fully uncover the correlation between language queries and video clips. Besides, we propose two data augmentation strategies to increase the diversity of training samples. The experimental results demonstrate the effectiveness of our method. The final submission ranked first on the leaderboard.