# Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories

arXiv: https://arxiv.org/abs/2308.10562 · HTML: https://mlsanity.com/paper/2308.10562/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 121 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 30 of 36)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Grounded Situation Recognition](https://mlsanity.com/paper/2003.12058/) (2020)
- [SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!](https://mlsanity.com/paper/2008.03781/) (2020)
- [Learning to Act Properly: Predicting and Explaining Affordances from Images](https://mlsanity.com/paper/1712.07576/) (2017)
- [How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrieval](https://mlsanity.com/paper/1810.09617/) (2018)
- [Automatic Understanding of Image and Video Advertisements](https://mlsanity.com/paper/1707.03067/) (2017)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [From Facial Expression Recognition to Interpersonal Relation Prediction](https://mlsanity.com/paper/1609.06426/) (2016)
- [Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases](https://mlsanity.com/paper/2102.11965/) (2021)
- [Situation Recognition with Graph Neural Networks](https://mlsanity.com/paper/1708.04320/) (2017)
- [Combining Language and Vision with a Multimodal Skip-gram Model](https://mlsanity.com/paper/1501.02598/) (2015)
- [Learning Social Relation Traits from Face Images](https://mlsanity.com/paper/1509.03936/) (2015)
- [Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning](https://mlsanity.com/paper/1901.01703/) (2019)
- [Intentonomy: a Dataset and Study towards Human Intent Understanding](https://mlsanity.com/paper/2011.05558/) (2020)
- [Dual-Glance Model for Deciphering Social Relationships](https://mlsanity.com/paper/1708.00634/) (2017)
- [ADVISE: Symbolism and External Knowledge for Decoding Advertisements](https://mlsanity.com/paper/1711.06666/) (2017)
- [Preserving Semantic Neighborhoods for Robust Cross-modal Retrieval](https://mlsanity.com/paper/2007.08617/) (2020)
- [Learning the Latent "Look": Unsupervised Discovery of a Style-Coherent Embedding from Fashion Images](https://mlsanity.com/paper/1707.03376/) (2017)
- [A Domain Based Approach to Social Relation Recognition](https://mlsanity.com/paper/1704.06456/) (2017)
- [We Are Humor Beings: Understanding and Predicting Visual Humor](https://mlsanity.com/paper/1512.04407/) (2015)
- [Graph-Based Social Relation Reasoning](https://mlsanity.com/paper/2007.07453/) (2020)
- [Integrating Knowledge and Reasoning in Image Understanding](https://mlsanity.com/paper/1906.09954/) (2019)
- [An End-to-End Network for Generating Social Relationship Graphs](https://mlsanity.com/paper/1903.09784/) (2019)
- [Understanding and Mapping Natural Beauty](https://mlsanity.com/paper/1612.03142/) (2016)
- [Visual Social Relationship Recognition](https://mlsanity.com/paper/1812.05917/) (2018)

## Built on by (most cited first, 4 of 4)

- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [Stitching Gaps: Fusing Situated Perceptual Knowledge with Vision Transformers for High-Level Image Classification](https://mlsanity.com/paper/2402.19339/) (2024)
- [When concept-based XAI is imprecise: Do people distinguish between generalisations and misrepresentations?](https://mlsanity.com/paper/2506.17936/) (2025)
- [Interpretable EEG-to-Image Generation with Semantic Prompts](https://mlsanity.com/paper/2507.07157/) (2025)

## Abstract (arXiv)

The field of Computer Vision (CV) is increasingly shifting towards ``high-level'' visual sensemaking tasks, yet the exact nature of these tasks remains unclear and tacit. This survey paper addresses this ambiguity by systematically reviewing research on high-level visual understanding, focusing particularly on Abstract Concepts (ACs) in automatic image classification. Our survey contributes in three main ways: Firstly, it clarifies the tacit understanding of high-level semantics in CV through a multidisciplinary analysis, and categorization into distinct clusters, including commonsense, emotional, aesthetic, and inductive interpretative semantics. Secondly, it identifies and categorizes computer vision tasks associated with high-level visual sensemaking, offering insights into the diverse research areas within this domain. Lastly, it examines how abstract concepts such as values and ideologies are handled in CV, revealing challenges and opportunities in AC-based image classification. Notably, our survey of AC image classification tasks highlights persistent challenges, such as the limited efficacy of massive datasets and the importance of integrating supplementary information and mid-level features. We emphasize the growing relevance of hybrid AI systems in addressing the multifaceted nature of AC image classification tasks. Overall, this survey enhances our understanding of high-level visual reasoning in CV and lays the groundwork for future research endeavors.