Winning the ICCV'2021 VALUE Challenge: Task-aware Ensemble and Transfer Learning with Visual Concepts
Not on the map. Why?
Abstract (arXiv)
The VALUE (Video-And-Language Understanding Evaluation) benchmark is newly introduced to evaluate and analyze multi-modal representation learning algorithms on three video-and-language tasks: Retrieval, QA, and Captioning. The main objective of the VALUE challenge is to train a task-agnostic model that is simultaneously applicable for various tasks with different characteristics. This technical report describes our winning strategies for the VALUE challenge: 1) single model optimization, 2) transfer learning with visual concepts, and 3) task-aware ensemble. The first and third strategies are designed to address heterogeneous characteristics of each task, and the second one is to leverage rich and fine-grained visual information. We provide a detailed and comprehensive analysis with extensive experimental results. Based on our approach, we ranked first place on the VALUE and QA phases for the competition.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 13 Oct 2021.
Builds on
16 of 19 references.
References · 16
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
- 2020Language Models are Few-Shot Learnerscited by 32,320
- 2018SlowFast Networks for Video Recognitioncited by 1,864
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
- 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
- 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
- 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 413
- 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
- 2016Image Captioning with Semantic Attentioncited by 260
4 more
- 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
- 2020What is More Likely to Happen Next? Video-and-Language Future Event Predictioncited by 61
- 2016End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answeringcited by 60
- 2020VIOLIN: A Large-Scale Dataset for Video-and-Language Inferencecited by 41
Built on by
Citing papers per year: 2022: 2.
2 papers.