Dialogue-to-Video Retrieval
Not on the map. Why?
Abstract (arXiv)
Recent years have witnessed an increasing amount of dialogue/conversation on the web especially on social media. That inspires the development of dialogue-based retrieval, in which retrieving videos based on dialogue is of increasing interest for recommendation systems. Different from other video retrieval tasks, dialogue-to-video retrieval uses structured queries in the form of user-generated dialogue as the search descriptor. We present a novel dialogue-to-video retrieval system, incorporating structured conversational information. Experiments conducted on the AVSD dataset show that our proposed approach using plain-text queries improves over the previous counterpart model by 15.8% on R@1. Furthermore, our approach using dialogue as a query, improves retrieval performance by 4.2%, 6.2%, 8.6% on R@1, R@5 and R@10 and outperforms the state-of-the-art model by 0.7%, 3.6% and 6.0% on R@1, R@5 and R@10 respectively.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 23 Mar 2023.
Builds on
24 of 33 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
- 2017Localizing Moments in Video with Natural Languagecited by 663
- 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
12 more
- 2020Multi-modal Transformer for Video Retrievalcited by 395
- 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
- 2015Sequence to Sequence -- Video to Textcited by 158
- 2019Audio-Visual Scene-Aware Dialogcited by 123
- 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
- 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
- 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
- 2021MMChat: Multi-Modal Chat Dataset on Social Mediacited by 30
- 2021Towards Diverse Paragraph Captioning for Untrimmed Videoscited by 22
- 2022Learning to Retrieve Videos by Asking Questionscited by 18
- 2019Interactive Video Retrieval with Dialogcited by 9
- 2023Improving Video Retrieval by Adaptive Margincited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.