mlsanityData to 14 Oct 2026

Fusion of Detected Objects in Text for Visual Question Answering

arXiv 1908.05054 ↗First posted 14 Aug 2019cs.CL cs.CV cs.LGcited by 74

Not on the map. Why?

Abstract (arXiv)

To advance models of multimodal context, we introduce a simple yet powerful neural architecture for data that combines vision and natural language. The "Bounding Boxes in Text Transformer" (B2T2) also leverages referential information binding words to portions of the image in a single unified architecture. B2T2 is highly effective on the Visual Commonsense Reasoning benchmark (https://visualcommonsense.com), achieving a new state-of-the-art with a 25% relative reduction in error rate compared to published baselines and obtaining the best performance to date on the public leaderboard (as of May 22, 2019). A detailed ablation analysis shows that the early integration of the visual features into the text analysis is key to the effectiveness of the new architecture. A reference implementation of our models is provided (https://github.com/google-research/language/tree/master/language/question_answering/b2t2).

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 29; 2022: 24; 2023: 13; 2024: 7; 2025: 1.

74 papers.

Sort:

By area:

  1. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  2. 2021Transformers in Vision: A Surveycited by 1,013
  3. 2022Multimodal Learning with Transformers: A Surveycited by 400
  4. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  5. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  6. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
  7. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  8. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  9. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  10. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  11. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  12. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
38 more
  1. 2021Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Taskscited by 125
  2. 2021UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modelingcited by 122
  3. 2021From Two to One: A New Scene Text Recognizer with Visual Language Modeling Networkcited by 97
  4. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  5. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  6. 2021Kaleido-BERT: Vision-Language Pre-training on Fashion Domaincited by 67
  7. 2021PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D Worldcited by 47
  8. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  9. 2022Contrastive Language-Image Pre-Training with Knowledge Graphscited by 34
  10. 2022Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Informationcited by 31
  11. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  12. 2022PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Modelscited by 30
  13. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  14. 2022A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketchcited by 28
  15. 2022PreSTU: Pre-Training for Scene-Text Understandingcited by 27
  16. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  17. 2022Visual-Language Navigation Pretraining via Prompt-based Environmental Self-explorationcited by 25
  18. 2023UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Modelscited by 25
  19. 2021Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledgecited by 24
  20. 2022Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Taskscited by 21
  21. 2021Knowledge Perceived Multi-modal Pretraining in E-commercecited by 18
  22. 2023Probabilistic Prompt Learning for Dense Predictioncited by 17
  23. 2021Detector-Free Weakly Supervised Grounding by Separationcited by 16
  24. 2021Auto-Parsing Network for Image Captioning and Visual Question Answeringcited by 16
  25. 2021Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Visioncited by 14
  26. 2022Masked Vision-Language Transformer in Fashioncited by 14
  27. 2024Demonstrating and Reducing Shortcuts in Vision-Language Representation Learningcited by 14
  28. 2021Audio-Visual Transformer Based Crowd Countingcited by 12
  29. 2022Learning to Evaluate Performance of Multi-modal Semantic Localizationcited by 12
  30. 2022A survey on knowledge-enhanced multimodal learningcited by 11
  31. 2023Deeply Coupled Cross-Modal Prompt Learningcited by 11
  32. 2024Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learningcited by 9
  33. 2021Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generationcited by 8
  34. 2023Separate and Locate: Rethink the Text in Text-based Visual Question Answeringcited by 7
  35. 2021Multi-stage Pre-training over Simplified Multimodal Pre-training Modelscited by 6
  36. 2022Learning by Hallucinating: Vision-Language Pre-training with Weak Supervisioncited by 6
  37. 2024Memory-Augmented Multimodal LLMs for Surgical VQA via Self-Contained Inquirycited by 6
  38. 2022Training and challenging models for text-guided fashion image retrievalcited by 5

50 most cited of 74 All 74 (JSON)