mlsanityData to 7 Oct 2026

Unbiased Scene Graph Generation from Biased Training

arXiv 2002.11949 ↗First posted 27 Feb 2020cs.CV cs.LGcited by 355

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Today's scene graph generation (SGG) task is still far from practical, mainly due to the severe training bias, e.g., collapsing diverse "human walk on / sit on / lay on beach" into "human on beach". Given such SGG, the down-stream tasks such as VQA can hardly infer better scene structures than merely a bag of objects. However, debiasing in SGG is not trivial because traditional debiasing methods cannot distinguish between the good and bad bias, e.g., good context prior (e.g., "person read book" rather than "eat") and bad long-tailed bias (e.g., "near" dominating "behind / in front of"). In this paper, we present a novel SGG framework based on causal inference but not the conventional likelihood. We first build a causal graph for SGG, and perform traditional biased training with the graph. Then, we propose to draw the counterfactual causality from the trained graph to infer the effect from the bad bias, which should be removed. In particular, we use Total Direct Effect (TDE) as the proposed final predicate score for unbiased SGG. Note that our framework is agnostic to any SGG model and thus can be widely applied in the community who seeks unbiased predictions. By using the proposed Scene Graph Diagnosis toolkit on the SGG benchmark Visual Genome and several prevailing models, we observed significant improvements over the previous state-of-the-art methods.

Key facts. Cited by 355 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 59; 2022: 77; 2023: 88; 2024: 61; 2025: 43; 2026: 27.

355 papers.

Sort:

By area:

  1. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  2. 2021Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generationcited by 137
  3. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  4. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  5. 2021Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identificationcited by 114
  6. 2022Panoptic Scene Graph Generationcited by 114
  7. 2021Causal Attention for Vision-Language Taskscited by 111
  8. 2021Energy-Based Learning for Scene Graph Generationcited by 104
  9. 2021SGTR: End-to-end Scene Graph Generation with Transformercited by 102
  10. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 101
  11. 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
  12. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
38 more
  1. 2021Semantic Communications: Principles and Challengescited by 90
  2. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  3. 2022Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoningcited by 78
  4. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  5. 2022Scene Graph Generation: A Comprehensive Surveycited by 73
  6. 2021Recovering the Unbiased Scene Graphs from the Biased Onescited by 71
  7. 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
  8. 2021Learning of Visual Relations: The Devil is in the Tailscited by 65
  9. 2021Learning to Generate Scene Graph from Natural Language Supervisioncited by 64
  10. 2022The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generationcited by 60
  11. 2021Fully Convolutional Scene Graph Generationcited by 57
  12. 2022Fine-Grained Scene Graph Generation with Data Transfercited by 57
  13. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  14. 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
  15. 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustmentcited by 51
  16. 2023Equivariant Similarity for Vision-Language Foundation Modelscited by 51
  17. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  18. 2024EGTR: Extracting Graph from Transformer for Scene Graph Generationcited by 45
  19. 2021Structured Sparse R-CNN for Direct Scene Graph Generationcited by 44
  20. 2022Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completioncited by 44
  21. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 44
  22. 2024From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Modelscited by 43
  23. 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
  24. 2023VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloudcited by 42
  25. 2023Panoptic Scene Graph Generation with Semantics-Prototype Learningcited by 42
  26. 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
  27. 2023Unbiased Scene Graph Generation in Videoscited by 38
  28. 2023Prototype-based Embedding Network for Scene Graph Generationcited by 37
  29. 2021Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generationcited by 35
  30. 2022Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Taskcited by 35
  31. 2023Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open Worldcited by 35
  32. 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
  33. 2023Context De-confounded Emotion Recognitioncited by 34
  34. 2021In Defense of Scene Graphs for Image Captioningcited by 31
  35. 2024STAR: A First-Ever Dataset and A Large-Scale Benchmark for Scene Graph Generation in Large-Size Satellite Imagerycited by 31
  36. 2022Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysiscited by 30
  37. 2022PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Modelscited by 30
  38. 2023LLM4SGG: Large Language Models for Weakly Supervised Scene Graph Generationcited by 30

50 most cited of 355 All 355 (JSON)