# On Exploring Undetermined Relationships for Visual Relationship Detection

arXiv: https://arxiv.org/abs/1905.01595 · HTML: https://mlsanity.com/paper/1905.01595/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2019.

## Built on by (most cited first, 26 of 26)

- [RelTR: Relation Transformer for Scene Graph Generation](https://mlsanity.com/paper/2201.11460/) (2022)
- [Detecting Human-Object Interaction via Fabricated Compositional Learning](https://mlsanity.com/paper/2103.08214/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Structured Sparse R-CNN for Direct Scene Graph Generation](https://mlsanity.com/paper/2106.10815/) (2021)
- [One-Shot Affordance Detection](https://mlsanity.com/paper/2106.14747/) (2021)
- [Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries](https://mlsanity.com/paper/2105.02170/) (2021)
- [Learning Graph Neural Networks for Image Style Transfer](https://mlsanity.com/paper/2207.11681/) (2022)
- [RU-Net: Regularized Unrolling Network for Scene Graph Generation](https://mlsanity.com/paper/2205.01297/) (2022)
- [Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation](https://mlsanity.com/paper/2201.06794/) (2022)
- [HL-Net: Heterophily Learning Network for Scene Graph Generation](https://mlsanity.com/paper/2205.01316/) (2022)
- [BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation](https://mlsanity.com/paper/2109.05346/) (2021)
- [Adaptive Fine-Grained Predicates Learning for Scene Graph Generation](https://mlsanity.com/paper/2207.04602/) (2022)
- [ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection](https://mlsanity.com/paper/2109.04047/) (2021)
- [Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions](https://mlsanity.com/paper/2110.03562/) (2021)
- [Learning Efficient, Explainable and Discriminative Representations for Pulmonary Nodules Classification](https://mlsanity.com/paper/2101.07429/) (2021)
- [Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization](https://mlsanity.com/paper/2108.09041/) (2021)
- [Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection](https://mlsanity.com/paper/2207.11789/) (2022)
- [Skew Class-balanced Re-weighting for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2301.00351/) (2023)
- [AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generation](https://mlsanity.com/paper/2404.07788/) (2024)
- [Explore Contextual Information for 3D Scene Graph Generation](https://mlsanity.com/paper/2210.06240/) (2022)
- [Human Object Interaction Detection using Two-Direction Spatial Enhancement and Exclusive Object Prior](https://mlsanity.com/paper/2105.03089/) (2021)
- [BCTR: Bidirectional Conditioning Transformer for Scene Graph Generation](https://mlsanity.com/paper/2407.18715/) (2024)
- [Hyper-relationship Learning Network for Scene Graph Generation](https://mlsanity.com/paper/2202.07271/) (2022)
- [A Novel Neural-symbolic System under Statistical Relational Learning](https://mlsanity.com/paper/2309.08931/) (2023)
- [A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generation](https://mlsanity.com/paper/2505.23451/) (2025)
- [Dynamic Relation Inference via Verb Embeddings](https://mlsanity.com/paper/2503.13021/) (2025)

## Abstract (arXiv)

In visual relationship detection, human-notated relationships can be regarded as determinate relationships. However, there are still large amount of unlabeled data, such as object pairs with less significant relationships or even with no relationships. We refer to these unlabeled but potentially useful data as undetermined relationships. Although a vast body of literature exists, few methods exploit these undetermined relationships for visual relationship detection. In this paper, we explore the beneficial effect of undetermined relationships on visual relationship detection. We propose a novel multi-modal feature based undetermined relationship learning network (MF-URLN) and achieve great improvements in relationship detection. In detail, our MF-URLN automatically generates undetermined relationships by comparing object pairs with human-notated data according to a designed criterion. Then, the MF-URLN extracts and fuses features of object pairs from three complementary modals: visual, spatial, and linguistic modals. Further, the MF-URLN proposes two correlated subnetworks: one subnetwork decides the determinate confidence, and the other predicts the relationships. We evaluate the MF-URLN on two datasets: the Visual Relationship Detection (VRD) and the Visual Genome (VG) datasets. The experimental results compared with state-of-the-art methods verify the significant improvements made by the undetermined relationships, e.g., the top-50 relation detection recall improves from 19.5% to 23.9% on the VRD dataset.