# Detecting Visual Relationships with Deep Relational Networks

arXiv: https://arxiv.org/abs/1704.03114 · HTML: https://mlsanity.com/paper/1704.03114/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2017.

## Built on by (most cited first, 30 of 97)

- [SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences](https://mlsanity.com/paper/2103.14898/) (2021)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Panoptic Scene Graph Generation](https://mlsanity.com/paper/2207.11247/) (2022)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [Spatial-Temporal Transformer for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2107.12309/) (2021)
- [RelTR: Relation Transformer for Scene Graph Generation](https://mlsanity.com/paper/2201.11460/) (2022)
- [The All-Seeing Project V2: Towards General Relation Comprehension of the Open World](https://mlsanity.com/paper/2402.19474/) (2024)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2203.09811/) (2022)
- [HierVL: Learning Hierarchical Video-Language Embeddings](https://mlsanity.com/paper/2301.02311/) (2023)
- [Learning to Generate Scene Graph from Natural Language Supervision](https://mlsanity.com/paper/2109.02227/) (2021)
- [Fully Convolutional Scene Graph Generation](https://mlsanity.com/paper/2103.16083/) (2021)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [Learning Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2105.01017/) (2021)
- [Deep Learning Approaches on Image Captioning: A Review](https://mlsanity.com/paper/2201.12944/) (2022)
- [Continuous Scene Representations for Embodied AI](https://mlsanity.com/paper/2203.17251/) (2022)
- [Unified Graph Structured Models for Video Understanding](https://mlsanity.com/paper/2103.15662/) (2021)
- [Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries](https://mlsanity.com/paper/2105.02170/) (2021)
- [Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioning](https://mlsanity.com/paper/2108.02366/) (2021)
- [Visual Distant Supervision for Scene Graph Generation](https://mlsanity.com/paper/2103.15365/) (2021)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Devil's on the Edges: Selective Quad Attention for Scene Graph Generation](https://mlsanity.com/paper/2304.03495/) (2023)
- [Exploiting Scene Graphs for Human-Object Interaction Detection](https://mlsanity.com/paper/2108.08584/) (2021)
- [HiLo: Exploiting High Low Frequency Relations for Unbiased Panoptic Scene Graph Generation](https://mlsanity.com/paper/2303.15994/) (2023)
- [Vision Relation Transformer for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2308.09472/) (2023)
- [Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection](https://mlsanity.com/paper/2311.01755/) (2023)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Grounding](https://mlsanity.com/paper/2310.06214/) (2023)

## Abstract (arXiv)

Relationships among objects play a crucial role in image understanding. Despite the great success of deep learning techniques in recognizing individual objects, reasoning about the relationships among objects remains a challenging task. Previous methods often treat this as a classification problem, considering each type of relationship (e.g. "ride") or each distinct visual phrase (e.g. "person-ride-horse") as a category. Such approaches are faced with significant difficulties caused by the high diversity of visual appearance for each kind of relationships or the large number of distinct visual phrases. We propose an integrated framework to tackle this problem. At the heart of this framework is the Deep Relational Network, a novel formulation designed specifically for exploiting the statistical dependencies between objects and their relationships. On two large datasets, the proposed method achieves substantial improvement over state-of-the-art.