mlsanityData to 7 Oct 2026

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

arXiv 1901.06706 ↗First posted 20 Jan 2019cs.CVcited by 250

Not on the map. Why?

Cited by 79 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing visual reasoning datasets such as Visual Question Answering (VQA), often suffer from biases conditioned on the question, image or answer distributions. The recently proposed CLEVR dataset addresses these limitations and requires fine-grained reasoning but the dataset is synthetic and consists of similar objects and sentence structures across the dataset. In this paper, we introduce a new inference task, Visual Entailment (VE) - consisting of image-sentence pairs whereby a premise is defined by an image, rather than a natural language sentence as in traditional Textual Entailment tasks. The goal of a trained VE model is to predict whether the image semantically entails the text. To realize this task, we build a dataset SNLI-VE based on the Stanford Natural Language Inference corpus and Flickr30k dataset. We evaluate various existing VQA baselines and build a model called Explainable Visual Entailment (EVE) system to address the VE task. EVE achieves up to 71% accuracy and outperforms several other state-of-the-art VQA based models. Finally, we demonstrate the explainability of EVE through cross-modal attention visualizations. The SNLI-VE dataset is publicly available at https://github.com/ necla-ml/SNLI-VE.

Key facts. Cited by 250 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 68; 2023: 65; 2024: 46; 2025: 27; 2026: 13.

250 papers.

Sort:

By area:

  1. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  5. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  6. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  7. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  8. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  9. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  10. 2022Visual Spatial Reasoningcited by 288
  11. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  12. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
38 more
  1. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  2. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  3. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  4. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  5. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  6. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  7. 2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 142
  8. 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 134
  9. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  10. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  11. 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
  12. 2022MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuningcited by 109
  13. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  14. 2023Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Trainingcited by 91
  15. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  16. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  17. 2022CLiMB: A Continual Learning Benchmark for Vision-and-Language Taskscited by 68
  18. 2022Masked Vision and Language Modeling for Multi-modal Representation Learningcited by 66
  19. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  20. 2023What Matters in Training a GPT4-Style Language Model with Multimodal Inputs?cited by 64
  21. 2022Multi-modal Alignment using Representation Codebookcited by 63
  22. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  23. 2022MixGen: A New Multi-Modal Data Augmentationcited by 61
  24. 2022NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Taskscited by 60
  25. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 59
  26. 2023Large Language Models are Visual Reasoning Coordinatorscited by 58
  27. 2022BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learningcited by 56
  28. 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
  29. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  30. 2023Towards Language Models That Can See: Computer Vision Through the LENS of Natural Languagecited by 53
  31. 2021Playing Lottery Tickets with Vision and Languagecited by 45
  32. 2021UFO: A UniFied TransfOrmer for Vision-Language Representation Learningcited by 45
  33. 2023IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Modelscited by 45
  34. 2023Towards In-context Scene Understandingcited by 45
  35. 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attentioncited by 44
  36. 2023UnIVAL: Unified Model for Image, Video, Audio and Language Taskscited by 44
  37. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Modelscited by 44
  38. 2022You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Modelcited by 42

50 most cited of 250 All 250 (JSON)