# MAC: Mining Activity Concepts for Language-based Temporal Localization

arXiv: https://arxiv.org/abs/1811.08925 · HTML: https://mlsanity.com/paper/1811.08925/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2018.

## Built on by (most cited first, 30 of 84)

- [Context-aware Biaffine Localizing Network for Temporal Sentence Grounding](https://mlsanity.com/paper/2103.11555/) (2021)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding](https://mlsanity.com/paper/2109.04872/) (2021)
- [Boundary Proposal Network for Two-Stage Natural Language Video Localization](https://mlsanity.com/paper/2103.08109/) (2021)
- [Knowing Where to Focus: Event-aware Transformer for Video Grounding](https://mlsanity.com/paper/2308.06947/) (2023)
- [Video Corpus Moment Retrieval with Contrastive Learning](https://mlsanity.com/paper/2105.06247/) (2021)
- [TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2401.02309/) (2024)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [On Pursuit of Designing Multi-modal Transformer for Video Grounding](https://mlsanity.com/paper/2109.06085/) (2021)
- [Memory-Guided Semantic Learning Network for Temporal Sentence Grounding](https://mlsanity.com/paper/2201.00454/) (2022)
- [Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video](https://mlsanity.com/paper/2203.06667/) (2022)
- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metric](https://mlsanity.com/paper/2101.09028/) (2021)
- [Weakly Supervised Temporal Adjacent Network for Language Grounding](https://mlsanity.com/paper/2106.16136/) (2021)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Grounding](https://mlsanity.com/paper/2203.02966/) (2022)
- [Reducing the Vision and Language Bias for Temporal Sentence Grounding](https://mlsanity.com/paper/2207.13457/) (2022)
- [Correlation-Guided Query-Dependency Calibration for Video Temporal Grounding](https://mlsanity.com/paper/2311.08835/) (2023)
- [Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval](https://mlsanity.com/paper/2407.15051/) (2024)
- [Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Grounding](https://mlsanity.com/paper/2103.16848/) (2021)
- [A Survey on Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2109.08039/) (2021)
- [BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2312.00083/) (2023)
- [Support-Set Based Cross-Supervision for Video Grounding](https://mlsanity.com/paper/2108.10576/) (2021)
- [Natural Language Video Localization with Learnable Moment Proposals](https://mlsanity.com/paper/2109.10678/) (2021)
- [Relation-aware Video Reading Comprehension for Temporal Language Grounding](https://mlsanity.com/paper/2110.05717/) (2021)
- [Hypotheses Tree Building for One-Shot Temporal Sentence Localization](https://mlsanity.com/paper/2301.01871/) (2023)
- [Video Activity Localisation with Uncertainties in Temporal Boundary](https://mlsanity.com/paper/2206.12923/) (2022)
- [Localizing Moments in Long Video Via Multimodal Guidance](https://mlsanity.com/paper/2302.13372/) (2023)
- [FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Grounding](https://mlsanity.com/paper/2412.13441/) (2024)
- [Parallel Attention Network with Sequence Matching for Video Grounding](https://mlsanity.com/paper/2105.08481/) (2021)

## Abstract (arXiv)

We address the problem of language-based temporal localization in untrimmed videos. Compared to temporal localization with fixed categories, this problem is more challenging as the language-based queries not only have no pre-defined activity list but also may contain complex descriptions. Previous methods address the problem by considering features from video sliding windows and language queries and learning a subspace to encode their correlation, which ignore rich semantic cues about activities in videos and queries. We propose to mine activity concepts from both video and language modalities by applying the actionness score enhanced Activity Concepts based Localizer (ACL). Specifically, the novel ACL encodes the semantic concepts from verb-obj pairs in language queries and leverages activity classifiers' prediction scores to encode visual concepts. Besides, ACL also has the capability to regress sliding windows as localization results. Experiments show that ACL significantly outperforms state-of-the-arts under the widely used metric, with more than 5% increase on both Charades-STA and TACoS datasets.