# Context Understanding in Computer Vision: A Survey

arXiv: https://arxiv.org/abs/2302.05011 · HTML: https://mlsanity.com/paper/2302.05011/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 100 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [Learning to Compose Dynamic Tree Structures for Visual Contexts](https://mlsanity.com/paper/1812.01880/) (2018)
- [Joint Detection and Identification Feature Learning for Person Search](https://mlsanity.com/paper/1604.01850/) (2016)
- [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016)
- [Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/1910.14303/) (2019)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [Modeling Visual Context is Key to Augmenting Object Detection Datasets](https://mlsanity.com/paper/1807.07428/) (2018)
- [Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction](https://mlsanity.com/paper/1909.05010/) (2019)
- [Semantic Relation Reasoning for Shot-Stable Few-Shot Object Detection](https://mlsanity.com/paper/2103.01903/) (2021)
- [Learning Context Graph for Person Search](https://mlsanity.com/paper/1904.01830/) (2019)

## Built on by (most cited first, 23 of 23)

- [Characterizing Photorealism and Artifacts in Diffusion Model-Generated Images](https://mlsanity.com/paper/2502.11989/) (2025)
- [CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Models](https://mlsanity.com/paper/2402.13607/) (2024)
- [Survey on video anomaly detection in dynamic scenes with moving cameras](https://mlsanity.com/paper/2308.07050/) (2023)
- [MOS: Modeling Object-Scene Associations in Generalized Category Discovery](https://mlsanity.com/paper/2503.12035/) (2025)
- [Common Inpainted Objects In-N-Out of Context](https://mlsanity.com/paper/2506.00721/) (2025)
- [Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models](https://mlsanity.com/paper/2311.16475/) (2023)
- [Computer Vision in the Food Industry: Accurate, Real-time, and Automatic Food Recognition with Pretrained MobileNetV2](https://mlsanity.com/paper/2405.11621/) (2024)
- [Context in object detection: a systematic literature review](https://mlsanity.com/paper/2503.23249/) (2025)
- [Position: Do Not Explain Vision Models Without Context](https://mlsanity.com/paper/2404.18316/) (2024)
- [Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation](https://mlsanity.com/paper/2412.06454/) (2024)
- [SceneGram: Conceptualizing and Describing Tangrams in Scene Context](https://mlsanity.com/paper/2506.11631/) (2025)
- [Caption Injection for Optimization in Generative Search Engine](https://mlsanity.com/paper/2511.04080/) (2025)
- [GMC: A General Framework of Multi-stage Context Learning and Utilization for Visual Detection Tasks](https://mlsanity.com/paper/2407.05566/) (2024)
- [Diverse Generation while Maintaining Semantic Coordination: A Diffusion-Based Data Augmentation Method for Object Detection](https://mlsanity.com/paper/2408.02891/) (2024)
- [Lost in Context: The Influence of Context on Feature Attribution Methods for Object Recognition](https://mlsanity.com/paper/2411.02833/) (2024)
- [Exploring Automated Recognition of Instructional Activity and Discourse from Multimodal Classroom Data](https://mlsanity.com/paper/2512.00087/) (2025)
- [Learning Scene Context Without Images](https://mlsanity.com/paper/2311.10998/) (2023)
- [A Concise Tiling Strategy for Preserving Spatial Context in Earth Observation Imagery](https://mlsanity.com/paper/2404.10927/) (2024)
- [A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attention](https://mlsanity.com/paper/2407.18673/) (2024)
- [ML Mule: Mobile-Driven Context-Aware Collaborative Learning](https://mlsanity.com/paper/2501.07536/) (2025)
- [Exploring the Application of Visual Question Answering (VQA) for Classroom Activity Monitoring](https://mlsanity.com/paper/2507.22369/) (2025)
- [Improving Classification of Occluded Objects through Scene Context](https://mlsanity.com/paper/2510.26681/) (2025)
- [OFBD: Object-Focused Background Debiasing for Long-Tailed Learning](https://mlsanity.com/paper/2609.37331/) (2026)

## Abstract (arXiv)

Contextual information plays an important role in many computer vision tasks, such as object detection, video action detection, image classification, etc. Recognizing a single object or action out of context could be sometimes very challenging, and context information may help improve the understanding of a scene or an event greatly. Appearance context information, e.g., colors or shapes of the background of an object can improve the recognition accuracy of the object in the scene. Semantic context (e.g. a keyboard on an empty desk vs. a keyboard next to a desktop computer ) will improve accuracy and exclude unrelated events. Context information that are not in the image itself, such as the time or location of an images captured, can also help to decide whether certain event or action should occur. Other types of context (e.g. 3D structure of a building) will also provide additional information to improve the accuracy. In this survey, different context information that has been used in computer vision tasks is reviewed. We categorize context into different types and different levels. We also review available machine learning models and image/video datasets that can employ context information. Furthermore, we compare context based integration and context-free integration in mainly two classes of tasks: image-based and video-based. Finally, this survey is concluded by a set of promising future directions in context learning and utilization.