# Visual Entailment Task for Visually-Grounded Language Learning

arXiv: https://arxiv.org/abs/1811.10582 · HTML: https://mlsanity.com/paper/1811.10582/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2018.

## Built on by (most cited first, 30 of 36)

- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning](https://mlsanity.com/paper/2104.03135/) (2021)
- [VLP: A Survey on Vision-Language Pre-training](https://mlsanity.com/paper/2202.09061/) (2022)
- [MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning](https://mlsanity.com/paper/2112.05253/) (2021)
- [Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training](https://mlsanity.com/paper/2106.13488/) (2021)
- [PuMer: Pruning and Merging Tokens for Efficient Vision Language Models](https://mlsanity.com/paper/2305.17530/) (2023)
- [MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model](https://mlsanity.com/paper/2210.05335/) (2022)
- [VideoCon: Robust Video-Language Alignment via Contrast Captions](https://mlsanity.com/paper/2311.10111/) (2023)
- [Vision-Language Intelligence: Tasks, Representation Learning, and Large Models](https://mlsanity.com/paper/2203.01922/) (2022)
- [Are Deep Neural Networks SMARTer than Second Graders?](https://mlsanity.com/paper/2212.09993/) (2022)
- [Vision+X: A Survey on Multimodal Learning in the Light of Data](https://mlsanity.com/paper/2210.02884/) (2022)
- [Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment](https://mlsanity.com/paper/2312.03766/) (2023)
- [A Survey on Quality Metrics for Text-to-Image Generation](https://mlsanity.com/paper/2403.11821/) (2024)
- [HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts](https://mlsanity.com/paper/2404.17507/) (2024)
- [SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning](https://mlsanity.com/paper/2112.08587/) (2021)
- [HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks](https://mlsanity.com/paper/2203.03878/) (2022)
- [A survey on knowledge-enhanced multimodal learning](https://mlsanity.com/paper/2211.12328/) (2022)
- [VELOCITI: Benchmarking Video-Language Compositional Reasoning with Strict Entailment](https://mlsanity.com/paper/2406.10889/) (2024)
- [Good Questions Help Zero-Shot Image Reasoning](https://mlsanity.com/paper/2312.01598/) (2023)
- [Tackling Vision Language Tasks Through Learning Inner Monologues](https://mlsanity.com/paper/2308.09970/) (2023)
- [Transferring Knowledge from Large Foundation Models to Small Downstream Models](https://mlsanity.com/paper/2406.07337/) (2024)
- [MLP Architectures for Vision-and-Language Modeling: An Empirical Study](https://mlsanity.com/paper/2112.04453/) (2021)
- [Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense](https://mlsanity.com/paper/2211.05895/) (2022)
- [AlignVE: Visual Entailment Recognition Based on Alignment Relations](https://mlsanity.com/paper/2211.08736/) (2022)
- [MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion](https://mlsanity.com/paper/2403.10568/) (2024)
- [3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset](https://mlsanity.com/paper/2404.18413/) (2024)
- [Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks](https://mlsanity.com/paper/2208.10758/) (2022)
- [An Empirical Study on the Language Modal in Visual Question Answering](https://mlsanity.com/paper/2305.10143/) (2023)
- [The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges](https://mlsanity.com/paper/2303.02411/) (2023)
- [Artificial-Spiking Hierarchical Networks for Vision-Language Representation Learning](https://mlsanity.com/paper/2308.09455/) (2023)

## Abstract (arXiv)

We introduce a new inference task - Visual Entailment (VE) - which differs from traditional Textual Entailment (TE) tasks whereby a premise is defined by an image, rather than a natural language sentence as in TE tasks. A novel dataset SNLI-VE (publicly available at https://github.com/necla-ml/SNLI-VE) is proposed for VE tasks based on the Stanford Natural Language Inference corpus and Flickr30k. We introduce a differentiable architecture called the Explainable Visual Entailment model (EVE) to tackle the VE problem. EVE and several other state-of-the-art visual question answering (VQA) based models are evaluated on the SNLI-VE dataset, facilitating grounded language understanding and providing insights on how modern VQA based models perform.