# Generative Recall, Dense Reranking: Learning Multi-View Semantic IDs for Efficient Text-to-Video Retrieval

arXiv: https://arxiv.org/abs/2601.21193 · HTML: https://mlsanity.com/paper/2601.21193/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [A Neural Corpus Indexer for Document Retrieval](https://mlsanity.com/paper/2206.02743/) (2022)
- [Learning to Tokenize for Generative Retrieval](https://mlsanity.com/paper/2304.04171/) (2023)
- [DiffusionRet: Generative Text-Video Retrieval with Diffusion Model](https://mlsanity.com/paper/2303.09867/) (2023)
- [Generative Recommender with End-to-End Learnable Item Tokenization](https://mlsanity.com/paper/2409.05546/) (2024)
- [Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond](https://mlsanity.com/paper/2402.10805/) (2024)
- [ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation](https://mlsanity.com/paper/2502.13581/) (2025)
- [MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent](https://mlsanity.com/paper/2511.18810/) (2025)
- [Inductive Generative Recommendation via Retrieval-based Speculation](https://mlsanity.com/paper/2410.02939/) (2024)
- [Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware Routing](https://mlsanity.com/paper/2503.10111/) (2025)
- [Multimodal Quantitative Language for Generative Recommendation](https://mlsanity.com/paper/2504.05314/) (2025)
- [Provable Ordering and Continuity in Vision-Language Pretraining for Generalizable Embodied Agents](https://mlsanity.com/paper/2502.01218/) (2025)

## Built on by (most cited first, 3 of 3)

- [Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction](https://mlsanity.com/paper/2606.24156/) (2026)
- [Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models](https://mlsanity.com/paper/2606.24165/) (2026)
- [Continual Video-MLLM Adaptation over Evolving Domains](https://mlsanity.com/paper/2607.18716/) (2026)

## Abstract (arXiv)

Text-to-Video Retrieval (TVR) is essential in video platforms. Dense retrieval with dual-modality encoders leads in accuracy, but its computation and storage scale poorly with corpus size. Thus, real-time large-scale applications adopt two-stage retrieval, where a fast recall model gathers a small candidate pool, which is reranked by an advanced dense retriever. Due to hugely reduced candidates, the reranking model can use any off-the-shelf dense retriever without hurting efficiency, meaning the recall model bounds two-stage TVR performance. Recently, generative retrieval (GR) replaces dense video embeddings with discrete semantic IDs and retrieves by decoding text queries into ID tokens. GR offers near-constant inference and storage complexity, and its semantic IDs capture high-level video features via quantization, making it ideal for quickly eliminating irrelevant candidates during recall. However, as a recall model in two-stage TVR, GR suffers from (i) semantic ambiguity, where each video satisfies diverse queries but is forced into one semantic ID; and (ii) cross-modal misalignment, as semantic IDs are solely derived from visual features without text supervision. We propose Generative Recall and Dense Reranking (GRDR), designing a novel GR method to uplift recalled candidate quality. GRDR assigns multiple semantic IDs to each video using a query-guided multi-view tokenizer exposing diverse semantic access paths, and jointly trains the tokenizer and generative retriever via a shared codebook to cast semantic IDs as the semantic bridge between texts and videos. At inference, trie-constrained decoding generates a compact candidate set reranked by a dense model for fine-grained matching. Experiments on TVR benchmarks show GRDR matches strong dense retrievers in accuracy while reducing index storage by an order of magnitude and accelerating up to 300× in full-corpus retrieval.