# L2C: Describing Visual Differences Needs Semantic Understanding of Individuals

arXiv: https://arxiv.org/abs/2102.01860 · HTML: https://mlsanity.com/paper/2102.01860/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Learning to Describe Differences Between Pairs of Similar Images](https://mlsanity.com/paper/1808.10584/) (2018)
- [Expressing Visual Relationships via Language](https://mlsanity.com/paper/1906.07689/) (2019)
- [No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling](https://mlsanity.com/paper/1804.09160/) (2018)
- [Diagnosing the Environment Bias in Vision-and-Language Navigation](https://mlsanity.com/paper/2005.03086/) (2020)
- [Iterative Visual Reasoning Beyond Convolutions](https://mlsanity.com/paper/1803.11189/) (2018)
- [Neural Naturalist: Generating Fine-Grained Image Comparisons](https://mlsanity.com/paper/1909.04101/) (2019)

## Built on by (most cited first, 8 of 8)

- [Learning Concise and Descriptive Attributes for Visual Recognition](https://mlsanity.com/paper/2308.03685/) (2023)
- [Image Difference Captioning with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2202.04298/) (2022)
- [Personalized Showcases: Generating Multi-Modal Explanations for Recommendations](https://mlsanity.com/paper/2207.00422/) (2022)
- [Image Retrieval from Contextual Descriptions](https://mlsanity.com/paper/2203.15867/) (2022)
- [OneDiff: A Generalist Model for Image Difference Captioning](https://mlsanity.com/paper/2407.05645/) (2024)
- [Rethinking the Reference-based Distinctive Image Captioning](https://mlsanity.com/paper/2207.11118/) (2022)
- [Improving Reference-based Distinctive Image Captioning with Contrastive Rewards](https://mlsanity.com/paper/2306.14259/) (2023)
- [TAB: Transformer Attention Bottlenecks enable User Intervention and Debugging in Vision-Language Models](https://mlsanity.com/paper/2412.18675/) (2024)

## Abstract (arXiv)

Recent advances in language and vision push forward the research of captioning a single image to describing visual differences between image pairs. Suppose there are two images, I_1 and I_2, and the task is to generate a description W_{1,2} comparing them, existing methods directly model { I_1, I_2 } -> W_{1,2} mapping without the semantic understanding of individuals. In this paper, we introduce a Learning-to-Compare (L2C) model, which learns to understand the semantic structures of these two images and compare them while learning to describe each one. We demonstrate that L2C benefits from a comparison between explicit semantic representations and single-image captions, and generalizes better on the new testing image pairs. It outperforms the baseline on both automatic evaluation and human evaluation for the Birds-to-Words dataset.