# Multimodal Lengthy Videos Retrieval Framework and Evaluation Metric

arXiv: https://arxiv.org/abs/2504.04572 · HTML: https://mlsanity.com/paper/2504.04572/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [Jasper and Stella: distillation of SOTA embedding models](https://mlsanity.com/paper/2412.19048/) (2024)
- [TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment](https://mlsanity.com/paper/2108.09980/) (2021)
- [Goldfish: Vision-Language Understanding of Arbitrarily Long Videos](https://mlsanity.com/paper/2407.12679/) (2024)
- [Multi-granularity Correspondence Learning from Long-term Noisy Videos](https://mlsanity.com/paper/2401.16702/) (2024)

## Built on by (most cited first, 1 of 1)

- [LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts](https://mlsanity.com/paper/2505.13928/) (2025)

## Abstract (arXiv)

Precise video retrieval requires multi-modal correlations to handle unseen vocabulary and scenes, becoming more complex for lengthy videos where models must perform effectively without prior training on a specific dataset. We introduce a unified framework that combines a visual matching stream and an aural matching stream with a unique subtitles-based video segmentation approach. Additionally, the aural stream includes a complementary audio-based two-stage retrieval mechanism that enhances performance on long-duration videos. Considering the complex nature of retrieval from lengthy videos and its corresponding evaluation, we introduce a new retrieval evaluation method specifically designed for long-video retrieval to support further research. We conducted experiments on the YouCook2 benchmark, showing promising retrieval performance.