mlsanityData to 7 Oct 2026

Simple Baselines for Interactive Video Retrieval with Questions and Answers

arXiv 2308.10402 ↗First posted 21 Aug 2023cs.CV cs.AI cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

To date, the majority of video retrieval systems have been optimized for a "single-shot" scenario in which the user submits a query in isolation, ignoring previous interactions with the system. Recently, there has been renewed interest in interactive systems to enhance retrieval, but existing approaches are complex and deliver limited gains in performance. In this work, we revisit this topic and propose several simple yet effective baselines for interactive video retrieval via question-answering. We employ a VideoQA model to simulate user interactions and show that this enables the productive study of the interactive retrieval task without access to ground truth dialogue data. Experiments on MSR-VTT, MSVD, and AVSD show that our framework using question-based interaction significantly improves the performance of text-based video retrieval systems.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 57 of its 77 references that are in the map. First posted on arXiv 21 Aug 2023.

Builds on

57 of 77 references.

References · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  9. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  10. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  11. 2017Localizing Moments in Video with Natural Languagecited by 660
  12. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
38 more
  1. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  2. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  3. 2020Multi-modal Transformer for Video Retrievalcited by 395
  4. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  5. 2016Visual Dialogcited by 377
  6. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  7. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  8. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  9. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  10. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  11. 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
  12. 2018A Joint Sequence Fusion Model for Video Question Answering and Retrievalcited by 208
  13. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 197
  14. 2015A Dataset for Movie Descriptioncited by 195
  15. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  16. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  17. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  18. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  19. 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
  20. 2019Audio-Visual Scene-Aware Dialogcited by 122
  21. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  22. 2018Motion-Appearance Co-Memory Networks for Video Question Answeringcited by 95
  23. 2022CenterCLIP: Token Clustering for Efficient Text-Video Retrievalcited by 86
  24. 2018Dialog-based Interactive Image Retrievalcited by 84
  25. 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
  26. 2020Location-aware Graph Convolutional Networks for Video Question Answeringcited by 80
  27. 2021Cross Modal Retrieval with Querybank Normalisationcited by 71
  28. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 70
  29. 2015Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answeringcited by 66
  30. 2016Dynamic Memory Networks for Visual and Textual Question Answeringcited by 62
  31. 2016End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answeringcited by 60
  32. 2016Generating Natural Questions About an Imagecited by 56
  33. 2014Improving zero-shot learning by mitigating the hubness problemcited by 53
  34. 2016Learning Language-Visual Embedding for Movie Understanding with Natural-Languagecited by 41
  35. 2019Information Maximizing Visual Question Generationcited by 40
  36. 2022Learning to Retrieve Videos by Asking Questionscited by 17
  37. 2018Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialogcited by 11
  38. 2018Learning Answer Embeddings for Visual Question Answeringcited by 11

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 6.

12 papers.

  1. 2025LLaVA-ReID: Selective Multi-image Questioner for Interactive Person Re-Identificationcited by 9
  2. 2025Quantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimizationcited by 5
  3. 2025Towards Efficient and Robust Moment Retrieval System: A Unified Framework for Multi-Granularity Models and Temporal Rerankingcited by 2
  4. 2024Multimodal LLM-based Query Paraphrasing for Video Searchcited by 1
  5. 2025Leveraging Auxiliary Information in Text-to-Video Retrieval: A Reviewcited by 1
  6. 2026DialogueVPR: Towards Conversational Visual Place Recognitioncited by 1
  7. 2025Composed Object Retrieval: Object-level Retrieval via Composed Expressionscited by 0
  8. 2026ARGOS: Who, Where, and When in Agentic Multi-Camera Person Searchcited by 0
  9. 2026A Sanity Check on Composed Image Retrievalcited by 0
  10. 2026Understanding the Performance Plateau in Text-to-Video Retrieval: A Comprehensive Empirical and Linguistic Analysiscited by 0
  11. 2026ADEPT: An Entropy-Driven Dual-Strategy Agent for Interactive Video Retrievalcited by 0
  12. 2026What Should We Ask Next? Retrieval-Aware Question Learning for Interactive ReIDcited by 0