# Cross-modal Contrastive Learning with Asymmetric Co-attention Network for Video Moment Retrieval

arXiv: https://arxiv.org/abs/2312.07435 · HTML: https://mlsanity.com/paper/2312.07435/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Local-Global Video-Text Interactions for Temporal Grounding](https://mlsanity.com/paper/2004.07514/) (2020)
- [mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections](https://mlsanity.com/paper/2205.12005/) (2022)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization](https://mlsanity.com/paper/2008.01403/) (2020)
- [Video Corpus Moment Retrieval with Contrastive Learning](https://mlsanity.com/paper/2105.06247/) (2021)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [End-to-end Multi-modal Video Temporal Grounding](https://mlsanity.com/paper/2107.05624/) (2021)
- [Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videos](https://mlsanity.com/paper/2008.02448/) (2020)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [A Survey on Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2109.08039/) (2021)
- [VLMAE: Vision-Language Masked Autoencoder](https://mlsanity.com/paper/2208.09374/) (2022)

## Built on by (most cited first, 4 of 4)

- [LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2501.10787/) (2025)
- [Multi-Scale Contrastive Learning for Video Temporal Grounding](https://mlsanity.com/paper/2412.07157/) (2024)
- [HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling](https://mlsanity.com/paper/2510.23043/) (2025)
- [ClipTBP: Clip-Pair based Temporal Boundary Prediction with Boundary-Aware Learning for Moment Retrieval](https://mlsanity.com/paper/2604.27591/) (2026)

## Abstract (arXiv)

Video moment retrieval is a challenging task requiring fine-grained interactions between video and text modalities. Recent work in image-text pretraining has demonstrated that most existing pretrained models suffer from information asymmetry due to the difference in length between visual and textual sequences. We question whether the same problem also exists in the video-text domain with an auxiliary need to preserve both spatial and temporal information. Thus, we evaluate a recently proposed solution involving the addition of an asymmetric co-attention network for video grounding tasks. Additionally, we incorporate momentum contrastive loss for robust, discriminative representation learning in both modalities. We note that the integration of these supplementary modules yields better performance compared to state-of-the-art models on the TACoS dataset and comparable results on ActivityNet Captions, all while utilizing significantly fewer parameters with respect to baseline.