# Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection

arXiv: https://arxiv.org/abs/1703.03054 · HTML: https://mlsanity.com/paper/1703.03054/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2017.

## Built on by (most cited first, 28 of 28)

- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Spatial-Temporal Transformer for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2107.12309/) (2021)
- [RelTR: Relation Transformer for Scene Graph Generation](https://mlsanity.com/paper/2201.11460/) (2022)
- [Weakly Supervised Object Localization and Detection: A Survey](https://mlsanity.com/paper/2104.07918/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval](https://mlsanity.com/paper/2203.14817/) (2022)
- [Coarse-to-Fine Reasoning for Visual Question Answering](https://mlsanity.com/paper/2110.02526/) (2021)
- [BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation](https://mlsanity.com/paper/2109.05346/) (2021)
- [Adaptive Fine-Grained Predicates Learning for Scene Graph Generation](https://mlsanity.com/paper/2207.04602/) (2022)
- [Beyond MOT: Semantic Multi-Object Tracking](https://mlsanity.com/paper/2403.05021/) (2024)
- [Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey](https://mlsanity.com/paper/2205.10766/) (2022)
- [Probabilistic Debiasing of Scene Graphs](https://mlsanity.com/paper/2211.06444/) (2022)
- [2.5D Visual Relationship Detection](https://mlsanity.com/paper/2104.12727/) (2021)
- [Neural Message Passing for Visual Relationship Detection](https://mlsanity.com/paper/2208.04165/) (2022)
- [State-Aware Variational Thompson Sampling for Deep Q-Networks](https://mlsanity.com/paper/2102.03719/) (2021)
- [Tackling the Challenges in Scene Graph Generation with Local-to-Global Interactions](https://mlsanity.com/paper/2106.08543/) (2021)
- [Representing Prior Knowledge Using Randomly, Weighted Feature Networks for Visual Relationship Detection](https://mlsanity.com/paper/2111.10686/) (2021)
- [ReCon1M:A Large-scale Benchmark Dataset for Relation Comprehension in Remote Sensing Imagery](https://mlsanity.com/paper/2406.06028/) (2024)
- [Scene Graph Modification as Incremental Structure Expanding](https://mlsanity.com/paper/2209.09093/) (2022)
- [Sketch Down the FLOPs: Towards Efficient Networks for Human Sketch](https://mlsanity.com/paper/2505.23763/) (2025)
- [Adaptive Remote Sensing Image Attribute Learning for Active Object Detection](https://mlsanity.com/paper/2101.06438/) (2021)
- [A Reinforcement Learning Approach for Sequential Spatial Transformer Networks](https://mlsanity.com/paper/2106.14295/) (2021)
- [What and When to Look?: Temporal Span Proposal Network for Video Relation Detection](https://mlsanity.com/paper/2107.07154/) (2021)
- [Improving Object Detection and Attribute Recognition by Feature Entanglement Reduction](https://mlsanity.com/paper/2108.11501/) (2021)
- [Towards Practicality of Sketch-Based Visual Understanding](https://mlsanity.com/paper/2210.15146/) (2022)
- [SGDraw: Scene Graph Drawing Interface Using Object-Oriented Representation](https://mlsanity.com/paper/2211.16697/) (2022)
- [Multi-modal reward for visual relationships-based image captioning](https://mlsanity.com/paper/2303.10766/) (2023)

## Abstract (arXiv)

Despite progress in visual perception tasks such as image classification and detection, computers still struggle to understand the interdependency of objects in the scene as a whole, e.g., relations between objects or their attributes. Existing methods often ignore global context cues capturing the interactions among different object instances, and can only recognize a handful of types by exhaustively training individual detectors for all possible relationships. To capture such global interdependency, we propose a deep Variation-structured Reinforcement Learning (VRL) framework to sequentially discover object relationships and attributes in the whole image. First, a directed semantic action graph is built using language priors to provide a rich and compact representation of semantic correlations between object categories, predicates, and attributes. Next, we use a variation-structured traversal over the action graph to construct a small, adaptive action set for each step based on the current state and historical actions. In particular, an ambiguity-aware object mining scheme is used to resolve semantic ambiguity among object categories that the object detector fails to distinguish. We then make sequential predictions using a deep RL framework, incorporating global context cues and semantic embeddings of previously extracted phrases in the state vector. Our experiments on the Visual Relationship Detection (VRD) dataset and the large-scale Visual Genome dataset validate the superiority of VRL, which can achieve significantly better detection results on datasets involving thousands of relationship and attribute types. We also demonstrate that VRL is able to predict unseen types embedded in our action graph by learning correlations on shared graph nodes.