Generalized Unbiased Scene Graph Generation
Not on the map. Why?
Abstract (arXiv)
Existing Unbiased Scene Graph Generation (USGG) methods only focus on addressing the predicate-level imbalance that high-frequency classes dominate predictions of rare ones, while overlooking the concept-level imbalance. Actually, even if predicates themselves are balanced, there is still a significant concept-imbalance within them due to the long-tailed distribution of contexts (i.e., subject-object combinations). This concept-level imbalance poses a more pervasive and challenging issue compared to the predicate-level imbalance since subject-object pairs are inherently complex in combinations. Hence, we introduce a novel research problem: Generalized Unbiased Scene Graph Generation (G-USGG), which takes into account both predicate-level and concept-level imbalance. To the end, we propose the Multi-Concept Learning (MCL) framework, which ensures a balanced learning process across rare/ uncommon/ common concepts. MCL first quantifies the concept-level imbalance across predicates in terms of different amounts of concepts, representing as multiple concept-prototypes within the same class. It then effectively learns concept-prototypes by applying the Concept Regularization (CR) technique. Furthermore, to achieve balanced learning over different concepts, we introduce the Balanced Prototypical Memory (BPM), which guides SGG models to generate balanced representations for concept-prototypes. Extensive experiments demonstrate the remarkable efficacy of our model-agnostic strategy in enhancing the performance of benchmark models on both VG-SGG and OI-SGG datasets, leading to new state-of-the-art achievements in two key aspects: predicate-level unbiased relation recognition and concept-level compositional generability.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 51 of its 73 references that are in the map. First posted on arXiv 9 Aug 2023.
Builds on
51 of 73 references.
References · 51
- 2017Attention Is All You Needcited by 59,932
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
- 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 413
- 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
- 2018Image Generation from Scene Graphscited by 324
- 2018Graph R-CNN for Scene Graph Generationcited by 296
- 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 258
- 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
- 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
- 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
38 more
- 2021Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bankcited by 125
- 2018Spreading vectors for similarity searchcited by 114
- 2021Energy-Based Learning for Scene Graph Generationcited by 104
- 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
- 2020PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generationcited by 75
- 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
- 2020CogTree: Cognition Tree Loss for Unbiased Scene Graph Generationcited by 62
- 2022The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generationcited by 60
- 2021Fully Convolutional Scene Graph Generationcited by 57
- 2022Fine-Grained Scene Graph Generation with Data Transfercited by 57
- 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustmentcited by 51
- 2023Regularized Vector Quantization for Tokenized Image Synthesiscited by 46
- 2021Structured Sparse R-CNN for Direct Scene Graph Generationcited by 44
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
- 2022BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptationcited by 39
- 2023Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphscited by 39
- 2023Prototype-based Embedding Network for Scene Graph Generationcited by 37
- 2020Structured Query-Based Image Retrieval Using Scene Graphscited by 33
- 2022Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptationcited by 33
- 2022Unsupervised Video Object Segmentation via Prototype Memory Networkcited by 30
- 2022RU-Net: Regularized Unrolling Network for Scene Graph Generationcited by 28
- 2022Interclass Prototype Relation for Few-Shot Segmentationcited by 25
- 2020Image-to-Image Retrieval by Learning Similarity between Scene Graphscited by 23
- 2023Devil's on the Edges: Selective Quad Attention for Scene Graph Generationcited by 23
- 2021Unconditional Scene Graph Generationcited by 22
- 2022Resistance Training using Prior Bias: toward Unbiased Scene Graph Generationcited by 22
- 2022HL-Net: Heterophily Learning Network for Scene Graph Generationcited by 22
- 2021Not All Relations are Equal: Mining Informative Labels for Scene Graph Generationcited by 21
- 2021SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoningcited by 16
- 2022Hierarchical Memory Learning for Fine-Grained Scene Graph Generationcited by 15
- 2020Generative Compositional Augmentations for Scene Graph Predictioncited by 14
- 2022Adaptive Fine-Grained Predicates Learning for Scene Graph Generationcited by 14
- 2021Semantic Compositional Learning for Low-shot Scene Graph Generationcited by 12
- 2021Predicate correlation learning for scene graph generationcited by 11
- 2021Few-Shot Object Detection by Attending to Per-Sample-Prototypecited by 9
- 2022A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTScited by 9
- 2022Probabilistic Debiasing of Scene Graphscited by 8
- 2022Learning To Generate Scene Graph from Head to Tailcited by 7
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2023: 3.
3 papers.