# The Demon is in Ambiguity: Revisiting Situation Recognition with Single Positive Multi-Label Learning

arXiv: https://arxiv.org/abs/2508.21816 · HTML: https://mlsanity.com/paper/2508.21816/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [CLIP-Event: Connecting Text and Images with Event Structures](https://mlsanity.com/paper/2201.05078/) (2022)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Grounded Situation Recognition](https://mlsanity.com/paper/2003.12058/) (2020)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [Large Loss Matters in Weakly Supervised Multi-Label Classification](https://mlsanity.com/paper/2206.03740/) (2022)
- [Situation Recognition with Graph Neural Networks](https://mlsanity.com/paper/1708.04320/) (2017)
- [Simple and Robust Loss Design for Multi-Label Learning with Missing Labels](https://mlsanity.com/paper/2112.07368/) (2021)
- [Acknowledging the Unknown for Multi-label Learning with Single Positive Labels](https://mlsanity.com/paper/2203.16219/) (2022)
- [Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels](https://mlsanity.com/paper/2303.13223/) (2023)
- [One Positive Label is Sufficient: Single-Positive Multi-Label Learning with Label Enhancement](https://mlsanity.com/paper/2206.00517/) (2022)
- [Collaborative Transformers for Grounded Situation Recognition](https://mlsanity.com/paper/2203.16518/) (2022)
- [Grounded Situation Recognition with Transformers](https://mlsanity.com/paper/2111.10135/) (2021)
- [Recurrent Models for Situation Recognition](https://mlsanity.com/paper/1703.06233/) (2017)
- [Ambiguous Images With Human Judgments for Robust Visual Event Classification](https://mlsanity.com/paper/2210.03102/) (2022)
- [Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification](https://mlsanity.com/paper/2304.01804/) (2023)
- [GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier](https://mlsanity.com/paper/1903.02585/) (2019)
- [ClipSitu: Effectively Leveraging CLIP for Conditional Predictions in Situation Recognition](https://mlsanity.com/paper/2307.00586/) (2023)
- [Context Does Matter: End-to-end Panoptic Narrative Grounding with Deformable Attention Refined Matching Network](https://mlsanity.com/paper/2310.16616/) (2023)
- [SpliceMix: A Cross-scale and Semantic Blending Augmentation Strategy for Multi-label Image Classification](https://mlsanity.com/paper/2311.15200/) (2023)

## Abstract (arXiv)

Context recognition (SR) is a fundamental task in computer vision that aims to extract structured semantic summaries from images by identifying key events and their associated entities. Specifically, given an input image, the model must first classify the main visual events (verb classification), then identify the participating entities and their semantic roles (semantic role labeling), and finally localize these entities in the image (semantic role localization). Existing methods treat verb classification as a single-label problem, but we show through a comprehensive analysis that this formulation fails to address the inherent ambiguity in visual event recognition, as multiple verb categories may reasonably describe the same image. This paper makes three key contributions: First, we reveal through empirical analysis that verb classification is inherently a multi-label problem due to the ubiquitous semantic overlap between verb categories. Second, given the impracticality of fully annotating large-scale datasets with multiple labels, we propose to reformulate verb classification as a single positive multi-label learning (SPMLL) problem - a novel perspective in SR research. Third, we design a comprehensive multi-label evaluation benchmark for SR that is carefully designed to fairly evaluate model performance in a multi-label setting. To address the challenges of SPMLL, we futher develop the Graph Enhanced Verb Multilayer Perceptron (GE-VerbMLP), which combines graph neural networks to capture label correlations and adversarial training to optimize decision boundaries. Extensive experiments on real-world datasets show that our approach achieves more than 3% MAP improvement while remaining competitive on traditional top-1 and top-5 accuracy metrics.