# Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships

arXiv: https://arxiv.org/abs/1807.00119 · HTML: https://mlsanity.com/paper/1807.00119/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2018.

## Built on by (most cited first, 30 of 41)

- [Learning Enriched Features for Fast Image Restoration and Enhancement](https://mlsanity.com/paper/2205.01649/) (2022)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networks](https://mlsanity.com/paper/2112.09791/) (2021)
- [EGTR: Extracting Graph from Transformer for Scene Graph Generation](https://mlsanity.com/paper/2404.02072/) (2024)
- [MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Grounding](https://mlsanity.com/paper/2403.03077/) (2024)
- [Context-Aware Transfer Attacks for Object Detection](https://mlsanity.com/paper/2112.03223/) (2021)
- [Zero-Query Transfer Attacks on Context-Aware Object Detectors](https://mlsanity.com/paper/2203.15230/) (2022)
- [Hallucination In Object Detection -- A Study In Visual Part Verification](https://mlsanity.com/paper/2106.02523/) (2021)
- [Circle Representation for Medical Object Detection](https://mlsanity.com/paper/2110.12093/) (2021)
- [GoG: Relation-aware Graph-over-Graph Network for Visual Dialog](https://mlsanity.com/paper/2109.08475/) (2021)
- [Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models](https://mlsanity.com/paper/2410.19635/) (2024)
- [Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex Scenes](https://mlsanity.com/paper/2108.08421/) (2021)
- [You Should Look at All Objects](https://mlsanity.com/paper/2207.07889/) (2022)
- [Deep Structured Instance Graph for Distilling Object Detectors](https://mlsanity.com/paper/2109.12862/) (2021)
- [Open-Vocabulary Object Detection via Scene Graph Discovery](https://mlsanity.com/paper/2307.03339/) (2023)
- [Spatial Reasoning for Few-Shot Object Detection](https://mlsanity.com/paper/2211.01080/) (2022)
- [Scenes and Surroundings: Scene Graph Generation using Relation Transformer](https://mlsanity.com/paper/2107.05448/) (2021)
- ["The Pedestrian next to the Lamppost" Adaptive Object Graphs for Better Instantaneous Mapping](https://mlsanity.com/paper/2204.02944/) (2022)
- [Evaluating Context for Deep Object Detectors](https://mlsanity.com/paper/2205.02887/) (2022)
- [Revisiting the Loss Weight Adjustment in Object Detection](https://mlsanity.com/paper/2103.09488/) (2021)
- [A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets](https://mlsanity.com/paper/2107.07927/) (2021)
- [Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligence](https://mlsanity.com/paper/2407.15320/) (2024)
- [Leveraging commonsense for object localisation in partial scenes](https://mlsanity.com/paper/2211.00562/) (2022)
- [Learning to See the Elephant in the Room: Self-Supervised Context Reasoning in Humans and AI](https://mlsanity.com/paper/2211.12817/) (2022)
- [LANDMARK: Language-guided Representation Enhancement Framework for Scene Graph Generation](https://mlsanity.com/paper/2303.01080/) (2023)
- [Boosting Open-Vocabulary Object Detection by Handling Background Samples](https://mlsanity.com/paper/2410.08645/) (2024)
- [Low Light Image Enhancement via Global and Local Context Modeling](https://mlsanity.com/paper/2101.00850/) (2021)
- [DCANet: Dense Context-Aware Network for Semantic Segmentation](https://mlsanity.com/paper/2104.02533/) (2021)
- [Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation](https://mlsanity.com/paper/2203.09160/) (2022)
- [Exploring Contextual Relationships for Cervical Abnormal Cell Detection](https://mlsanity.com/paper/2207.04693/) (2022)

## Abstract (arXiv)

Context is important for accurate visual recognition. In this work we propose an object detection algorithm that not only considers object visual appearance, but also makes use of two kinds of context including scene contextual information and object relationships within a single image. Therefore, object detection is regarded as both a cognition problem and a reasoning problem when leveraging these structured information. Specifically, this paper formulates object detection as a problem of graph structure inference, where given an image the objects are treated as nodes in a graph and relationships between the objects are modeled as edges in such graph. To this end, we present a so-called Structure Inference Network (SIN), a detector that incorporates into a typical detection framework (e.g. Faster R-CNN) with a graphical model which aims to infer object state. Comprehensive experiments on PASCAL VOC and MS COCO datasets indicate that scene context and object relationships truly improve the performance of object detection with more desirable and reasonable outputs.