# SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval

arXiv: https://arxiv.org/abs/2310.05241 · HTML: https://mlsanity.com/paper/2310.05241/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://mlsanity.com/paper/2005.00200/) (2020)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)
- [UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2203.12745/) (2022)
- [Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/1910.14303/) (2019)
- [W-TALC: Weakly-supervised Temporal Activity Localization and Classification](https://mlsanity.com/paper/1807.10418/) (2018)
- [Weakly Supervised Video Moment Retrieval From Text Queries](https://mlsanity.com/paper/1904.03282/) (2019)
- [Weakly-Supervised Video Moment Retrieval via Semantic Completion Network](https://mlsanity.com/paper/1911.08199/) (2019)
- [Weakly Supervised Dense Event Captioning in Videos](https://mlsanity.com/paper/1812.03849/) (2018)
- [VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/2008.10238/) (2020)
- [LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/1909.13784/) (2019)
- [Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation](https://mlsanity.com/paper/2107.11443/) (2021)
- [Weakly Supervised Temporal Adjacent Network for Language Grounding](https://mlsanity.com/paper/2106.16136/) (2021)
- [WSLLN: Weakly Supervised Natural Language Localization Networks](https://mlsanity.com/paper/1909.00239/) (2019)
- [Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation](https://mlsanity.com/paper/2203.02925/) (2022)
- [Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos](https://mlsanity.com/paper/2003.07048/) (2020)
- [Look Closer to Ground Better: Weakly-Supervised Temporal Grounding of Sentence in Video](https://mlsanity.com/paper/2001.09308/) (2020)
- [Regularized Two-Branch Proposal Networks for Weakly-Supervised Moment Retrieval in Videos](https://mlsanity.com/paper/2008.08257/) (2020)
- [Reinforcement Learning for Weakly Supervised Temporal Grounding of Natural Language in Untrimmed Videos](https://mlsanity.com/paper/2009.08614/) (2020)
- [Selective Query-guided Debiasing for Video Corpus Moment Retrieval](https://mlsanity.com/paper/2210.08714/) (2022)
- [You Need to Read Again: Multi-granularity Perception Network for Moment Retrieval in Videos](https://mlsanity.com/paper/2205.12886/) (2022)
- [Weak Supervision and Referring Attention for Temporal-Textual Association Learning](https://mlsanity.com/paper/2006.11747/) (2020)

## Built on by (most cited first, 13 of 13)

- [FRAG: Frequency Adapting Group for Diffusion Video Editing](https://mlsanity.com/paper/2406.06044/) (2024)
- [HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue](https://mlsanity.com/paper/2312.09736/) (2023)
- [TPC: Test-time Procrustes Calibration for Diffusion-based Human Image Animation](https://mlsanity.com/paper/2410.24037/) (2024)
- [Localizing Events in Videos with Multimodal Queries](https://mlsanity.com/paper/2406.10079/) (2024)
- [EtC: Temporal Boundary Expand then Clarify for Weakly Supervised Video Grounding with Multimodal Large Language Model](https://mlsanity.com/paper/2312.02483/) (2023)
- [Contrast-Unity for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2502.12917/) (2025)
- [Policy Learning from Large Vision-Language Model Feedback without Reward Modeling](https://mlsanity.com/paper/2507.23391/) (2025)
- [Occlusion-robust Stylization for Drawing-based 3D Animation](https://mlsanity.com/paper/2508.00398/) (2025)
- [Weakly Supervised Temporal Sentence Grounding via Positive Sample Mining](https://mlsanity.com/paper/2505.06557/) (2025)
- [Finding Optimal Video Moment without Training: Gaussian Boundary Optimization for Weakly Supervised Video Grounding](https://mlsanity.com/paper/2602.03071/) (2026)
- [Enhancing Weakly Supervised Video Grounding via Diverse Inference Strategies for Boundary and Prediction Selection](https://mlsanity.com/paper/2503.23181/) (2025)
- [Understanding the Performance Plateau in Text-to-Video Retrieval: A Comprehensive Empirical and Linguistic Analysis](https://mlsanity.com/paper/2605.00826/) (2026)
- [Multi-proposal Collaboration and Multi-task Training for Weakly-supervised Video Moment Retrieval](https://mlsanity.com/paper/2605.14838/) (2026)

## Abstract (arXiv)

Video moment retrieval aims to localize moments in video corresponding to a given language query. To avoid the expensive cost of annotating the temporal moments, weakly-supervised VMR (wsVMR) systems have been studied. For such systems, generating a number of proposals as moment candidates and then selecting the most appropriate proposal has been a popular approach. These proposals are assumed to contain many distinguishable scenes in a video as candidates. However, existing proposals of wsVMR systems do not respect the varying numbers of scenes in each video, where the proposals are heuristically determined irrespective of the video. We argue that the retrieval system should be able to counter the complexities caused by varying numbers of scenes in each video. To this end, we present a novel concept of a retrieval system referred to as Scene Complexity Aware Network (SCANet), which measures the `scene complexity' of multiple scenes in each video and generates adaptive proposals responding to variable complexities of scenes in each video. Experimental results on three retrieval benchmarks (i.e., Charades-STA, ActivityNet, TVR) achieve state-of-the-art performances and demonstrate the effectiveness of incorporating the scene complexity.