# Detecting out-of-context objects using contextual cues

arXiv: https://arxiv.org/abs/2202.05930 · HTML: https://mlsanity.com/paper/2202.05930/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks](https://mlsanity.com/paper/1909.01315/) (2019)
- [Discriminative Embeddings of Latent Variable Models for Structured Data](https://mlsanity.com/paper/1603.05629/) (2016)
- [The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation](https://mlsanity.com/paper/1904.03215/) (2019)
- [Modeling Visual Context is Key to Augmenting Object Detection Datasets](https://mlsanity.com/paper/1807.07428/) (2018)
- [GMNN: Graph Markov Neural Networks](https://mlsanity.com/paper/1905.06214/) (2019)
- [Putting visual object recognition in context](https://mlsanity.com/paper/1911.07349/) (2019)
- [When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes](https://mlsanity.com/paper/2104.02215/) (2021)
- [Seeing What Is Not There: Learning Context to Determine Where Objects Are Missing](https://mlsanity.com/paper/1702.07971/) (2017)
- [Identifying and Benchmarking Natural Out-of-Context Prediction Problems](https://mlsanity.com/paper/2110.13223/) (2021)

## Built on by (most cited first, 2 of 2)

- [Robust Context-Aware Object Recognition](https://mlsanity.com/paper/2510.00618/) (2025)
- [Bringing the Context Back into Object Recognition, Robustly](https://mlsanity.com/paper/2411.15933/) (2024)

## Abstract (arXiv)

This paper presents an approach to detect out-of-context (OOC) objects in an image. Given an image with a set of objects, our goal is to determine if an object is inconsistent with the scene context and detect the OOC object with a bounding box. In this work, we consider commonly explored contextual relations such as co-occurrence relations, the relative size of an object with respect to other objects, and the position of the object in the scene. We posit that contextual cues are useful to determine object labels for in-context objects and inconsistent context cues are detrimental to determining object labels for out-of-context objects. To realize this hypothesis, we propose a graph contextual reasoning network (GCRN) to detect OOC objects. GCRN consists of two separate graphs to predict object labels based on the contextual cues in the image: 1) a representation graph to learn object features based on the neighboring objects and 2) a context graph to explicitly capture contextual cues from the neighboring objects. GCRN explicitly captures the contextual cues to improve the detection of in-context objects and identify objects that violate contextual relations. In order to evaluate our approach, we create a large-scale dataset by adding OOC object instances to the COCO images. We also evaluate on recent OCD benchmark. Our results show that GCRN outperforms competitive baselines in detecting OOC objects and correctly detecting in-context objects.