# EVOQUER: Enhancing Temporal Grounding with Video-Pivoted BackQuery Generation

arXiv: https://arxiv.org/abs/2109.04600 · HTML: https://mlsanity.com/paper/2109.04600/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research](https://mlsanity.com/paper/1904.03493/) (2019)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [Local-Global Video-Text Interactions for Temporal Grounding](https://mlsanity.com/paper/2004.07514/) (2020)
- [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016)
- [Sequence to Sequence -- Video to Text](https://mlsanity.com/paper/1505.00487/) (2015)
- [To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression](https://mlsanity.com/paper/1804.07014/) (2018)
- [MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment](https://mlsanity.com/paper/1812.00087/) (2018)
- [Multilevel Language and Vision Integration for Text-to-Clip Retrieval](https://mlsanity.com/paper/1804.05113/) (2018)
- [Translating Videos to Natural Language Using Deep Recurrent Neural Networks](https://mlsanity.com/paper/1412.4729/) (2014)
- [Describing Videos by Exploiting Temporal Structure](https://mlsanity.com/paper/1502.08029/) (2015)
- [MAC: Mining Activity Concepts for Language-based Temporal Localization](https://mlsanity.com/paper/1811.08925/) (2018)
- [ExCL: Extractive Clip Localization Using Natural Language Descriptions](https://mlsanity.com/paper/1904.02755/) (2019)
- [Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attention](https://mlsanity.com/paper/1908.07236/) (2019)
- [Grounded Video Description](https://mlsanity.com/paper/1812.06587/) (2018)
- [Weakly-Supervised Video Moment Retrieval via Semantic Completion Network](https://mlsanity.com/paper/1911.08199/) (2019)
- [Read, Watch, and Move: Reinforcement Learning for Temporally Grounding Natural Language Descriptions in Videos](https://mlsanity.com/paper/1901.06829/) (2019)
- [Countering Language Drift via Visual Grounding](https://mlsanity.com/paper/1909.04499/) (2019)
- [LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/1909.13784/) (2019)
- [Video Captioning with Multi-Faceted Attention](https://mlsanity.com/paper/1612.00234/) (2016)
- [A Pipeline for Creative Visual Storytelling](https://mlsanity.com/paper/1807.08077/) (2018)
- [From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots](https://mlsanity.com/paper/1906.00872/) (2019)
- [A Multi-scale Multiple Instance Video Description Network](https://mlsanity.com/paper/1505.05914/) (2015)

## Built on by (most cited first, 1 of 1)

- [Text-Visual Prompting for Efficient 2D Temporal Video Grounding](https://mlsanity.com/paper/2303.04995/) (2023)

## Abstract (arXiv)

Temporal grounding aims to predict a time interval of a video clip corresponding to a natural language query input. In this work, we present EVOQUER, a temporal grounding framework incorporating an existing text-to-video grounding model and a video-assisted query generation network. Given a query and an untrimmed video, the temporal grounding model predicts the target interval, and the predicted video clip is fed into a video translation task by generating a simplified version of the input query. EVOQUER forms closed-loop learning by incorporating loss functions from both temporal grounding and query generation serving as feedback. Our experiments on two widely used datasets, Charades-STA and ActivityNet, show that EVOQUER achieves promising improvements by 1.05 and 1.31 at R@0.7. We also discuss how the query generation task could facilitate error analysis by explaining temporal grounding model behavior.