mlsanityData to 7 Oct 2026

Achieving Human Parity on Visual Question Answering

arXiv 2111.08896 ↗First posted 17 Nov 2021cs.CL cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

The Visual Question Answering (VQA) task utilizes both visual image and language analysis to answer a textual question with respect to an image. It has been a popular research topic with an increasing number of real-world applications in the last decade. This paper describes our recent research of AliceMind-MMU (ALIbaba's Collection of Encoder-decoders from Machine IntelligeNce lab of Damo academy - MultiMedia Understanding) that obtains similar or even slightly better results than human being does on VQA. This is achieved by systematically improving the VQA pipeline including: (1) pre-training with comprehensive visual and textual feature representation; (2) effective cross-modal interaction with learning to attend; and (3) A novel knowledge mining framework with specialized expert modules for the complex VQA task. Treating different types of visual questions with corresponding expertise needed plays an important role in boosting the performance of our VQA architecture up to the human level. An extensive set of experiments and analysis are conducted to demonstrate the effectiveness of the new research work.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 73 of its 105 references that are in the map. First posted on arXiv 17 Nov 2021.

Builds on

73 of 105 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  12. 2017Squeeze-and-Excitation Networkscited by 4,763
38 more
  1. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  10. 2015VQA: Visual Question Answeringcited by 2,489
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  13. 2019Towards VQA Models That Can Readcited by 1,696
  14. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  15. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  16. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  17. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  18. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  19. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  20. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  21. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  22. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  23. 2020Pre-Trained Image Processing Transformercited by 891
  24. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  25. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  26. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  27. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  28. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  29. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  30. 2018Two at Once: Enhancing Learning and Generalization Capacities via IBN-Netcited by 361
  31. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  32. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  33. 2019Scene Text Visual Question Answeringcited by 327
  34. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  35. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  36. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  37. 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
  38. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 0; 2025: 2.

5 papers.

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2023Adaptive loose optimization for robust question answeringcited by 2
  3. 2022One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Taskscited by 1
  4. 2025From Perception to Punchline: Empowering VLM with the Art of In-the-wild Memecited by 1
  5. 2025A Self-Supervised Mixture-of-Experts Framework for Multi-behavior Recommendationcited by 0