mlsanityData to 14 Oct 2026

VisualBERT: A Simple and Performant Baseline for Vision and Language

arXiv 1908.03557 ↗First posted 9 Aug 2019cs.CV cs.CL cs.LGcited by 1,296

Not on the map. Why?

Cited by 299 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks. VisualBERT consists of a stack of Transformer layers that implicitly align elements of an input text and regions in an associated input image with self-attention. We further propose two visually-grounded language model objectives for pre-training VisualBERT on image caption data. Experiments on four vision-and-language tasks including VQA, VCR, NLVR2, and Flickr30K show that VisualBERT outperforms or rivals with state-of-the-art models while being significantly simpler. Further analysis demonstrates that VisualBERT can ground elements of language to image regions without any explicit supervision and is even sensitive to syntactic relationships, tracking, for example, associations between verbs and image regions corresponding to their arguments.

Key facts. Cited by 1,296 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 203; 2022: 296; 2023: 330; 2024: 242; 2025: 161; 2026: 64.

1,296 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  8. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  9. 2021Grounded Language-Image Pre-trainingcited by 1,156
  10. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  11. 2021Transformers in Vision: A Surveycited by 1,013
  12. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
38 more
  1. 2021Multiscale Vision Transformerscited by 897
  2. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  3. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  4. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  5. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  6. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  7. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  8. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  9. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  10. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  11. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  12. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  13. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  14. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  15. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  16. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  17. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  18. 2021InfographicVQAcited by 437
  19. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  20. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  21. 2021A Survey of Transformerscited by 401
  22. 2022Multimodal Learning with Transformers: A Surveycited by 400
  23. 2021Unified Pre-training for Program Understanding and Generationcited by 366
  24. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  25. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  26. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  27. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  28. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 320
  29. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  30. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
  31. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  32. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  33. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  34. 2022Visual Spatial Reasoningcited by 293
  35. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  36. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
  37. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  38. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275

50 most cited of 1,296 All 1,296 (JSON)