# Putting visual object recognition in context

arXiv: https://arxiv.org/abs/1911.07349 · HTML: https://mlsanity.com/paper/1911.07349/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2019.

## Built on by (most cited first, 30 of 33)

- [Class-agnostic Object Detection with Multi-modal Transformer](https://mlsanity.com/paper/2111.11430/) (2021)
- [MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Grounding](https://mlsanity.com/paper/2403.03077/) (2024)
- [Discriminative Region-based Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2108.09301/) (2021)
- [When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes](https://mlsanity.com/paper/2104.02215/) (2021)
- [Context Understanding in Computer Vision: A Survey](https://mlsanity.com/paper/2302.05011/) (2023)
- [Look Twice: A Generalist Computational Model Predicts Return Fixations across Tasks and Species](https://mlsanity.com/paper/2101.01611/) (2021)
- [Efficient Zero-shot Visual Search via Target and Context-aware Transformer](https://mlsanity.com/paper/2211.13470/) (2022)
- [Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion](https://mlsanity.com/paper/2306.08889/) (2023)
- [Common Inpainted Objects In-N-Out of Context](https://mlsanity.com/paper/2506.00721/) (2025)
- [SpaceYOLO: A Human-Inspired Model for Real-time, On-board Spacecraft Feature Detection](https://mlsanity.com/paper/2302.00824/) (2023)
- [Spatial-context-aware deep neural network for multi-class image classification](https://mlsanity.com/paper/2111.12296/) (2021)
- [TDAM: Top-Down Attention Module for Contextually Guided Feature Selection in CNNs](https://mlsanity.com/paper/2111.13470/) (2021)
- [Evaluating Context for Deep Object Detectors](https://mlsanity.com/paper/2205.02887/) (2022)
- [Extreme Image Transformations Affect Humans and Machines Differently](https://mlsanity.com/paper/2212.13967/) (2022)
- [Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition](https://mlsanity.com/paper/2404.10904/) (2024)
- [Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex](https://mlsanity.com/paper/2406.16935/) (2024)
- [Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines](https://mlsanity.com/paper/2511.14440/) (2025)
- [Detecting out-of-context objects using contextual cues](https://mlsanity.com/paper/2202.05930/) (2022)
- [Improving generalization by mimicking the human visual diet](https://mlsanity.com/paper/2206.07802/) (2022)
- [Learning to See the Elephant in the Room: Self-Supervised Context Reasoning in Humans and AI](https://mlsanity.com/paper/2211.12817/) (2022)
- [PV2TEA: Patching Visual Modality to Textual-Established Information Extraction](https://mlsanity.com/paper/2306.01016/) (2023)
- [Resilience through Scene Context in Visual Referring Expression Generation](https://mlsanity.com/paper/2404.12289/) (2024)
- [Gradient Mask: Lateral Inhibition Mechanism Improves Performance in Artificial Neural Networks](https://mlsanity.com/paper/2208.06918/) (2022)
- [VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception](https://mlsanity.com/paper/2308.01525/) (2023)
- [Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning](https://mlsanity.com/paper/2605.09963/) (2026)
- [Context-driven Visual Object Recognition based on Knowledge Graphs](https://mlsanity.com/paper/2210.11233/) (2022)
- [Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap](https://mlsanity.com/paper/2211.13087/) (2022)
- [Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentation](https://mlsanity.com/paper/2211.14074/) (2022)
- [Prediction of Scene Plausibility](https://mlsanity.com/paper/2212.01470/) (2022)
- [Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2306.01929/) (2023)

## Abstract (arXiv)

Context plays an important role in visual recognition. Recent studies have shown that visual recognition networks can be fooled by placing objects in inconsistent contexts (e.g., a cow in the ocean). To model the role of contextual information in visual recognition, we systematically investigated ten critical properties of where, when, and how context modulates recognition, including the amount of context, context and object resolution, geometrical structure of context, context congruence, and temporal dynamics of contextual modulation. The tasks involved recognizing a target object surrounded with context in a natural image. As an essential benchmark, we conducted a series of psychophysics experiments where we altered one aspect of context at a time, and quantified recognition accuracy. We propose a biologically-inspired context-aware object recognition model consisting of a two-stream architecture. The model processes visual information at the fovea and periphery in parallel, dynamically incorporates object and contextual information, and sequentially reasons about the class label for the target object. Across a wide range of behavioral tasks, the model approximates human level performance without retraining for each task, captures the dependence of context enhancement on image properties, and provides initial steps towards integrating scene and object information for visual recognition. All source code and data are publicly available: https://github.com/kreimanlab/Put-In-Context.