# Learning Visual Commonsense for Robust Scene Graph Generation

arXiv: https://arxiv.org/abs/2006.09623 · HTML: https://mlsanity.com/paper/2006.09623/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 30 of 47)

- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Panoptic Scene Graph Generation](https://mlsanity.com/paper/2207.11247/) (2022)
- [Energy-Based Learning for Scene Graph Generation](https://mlsanity.com/paper/2103.02221/) (2021)
- [SGTR: End-to-end Scene Graph Generation with Transformer](https://mlsanity.com/paper/2112.12970/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Recovering the Unbiased Scene Graphs from the Biased Ones](https://mlsanity.com/paper/2107.02112/) (2021)
- [Fully Convolutional Scene Graph Generation](https://mlsanity.com/paper/2103.16083/) (2021)
- [Segmentation-grounded Scene Graph Generation](https://mlsanity.com/paper/2104.14207/) (2021)
- [Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning](https://mlsanity.com/paper/2502.12425/) (2025)
- [SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning](https://mlsanity.com/paper/2112.08587/) (2021)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [Semantic Compositional Learning for Low-shot Scene Graph Generation](https://mlsanity.com/paper/2108.08600/) (2021)
- [Scene Graph Expansion for Semantics-Guided Image Outpainting](https://mlsanity.com/paper/2205.02958/) (2022)
- [Can Transformers Capture Spatial Relations between Objects?](https://mlsanity.com/paper/2403.00729/) (2024)
- [Improving Scene Graph Classification by Exploiting Knowledge from Texts](https://mlsanity.com/paper/2102.04760/) (2021)
- [A deep learning model for data-driven discovery of functional connectivity](https://mlsanity.com/paper/2112.04013/) (2021)
- [Probabilistic Debiasing of Scene Graphs](https://mlsanity.com/paper/2211.06444/) (2022)
- [Adaptive Visual Scene Understanding: Incremental Scene Graph Generation](https://mlsanity.com/paper/2310.01636/) (2023)
- [Graph Laplacian for Semi-Supervised Learning](https://mlsanity.com/paper/2301.04956/) (2023)
- [On the Security Risks of Knowledge Graph Reasoning](https://mlsanity.com/paper/2305.02383/) (2023)
- [Explore Contextual Information for 3D Scene Graph Generation](https://mlsanity.com/paper/2210.06240/) (2022)
- [HAtt-Flow: Hierarchical Attention-Flow Mechanism for Group Activity Scene Graph Generation in Videos](https://mlsanity.com/paper/2312.07740/) (2023)
- [Evaluating the Quality of Finite Element Meshes with Machine Learning](https://mlsanity.com/paper/2107.10507/) (2021)
- [Is Object Detection Necessary for Human-Object Interaction Recognition?](https://mlsanity.com/paper/2107.13083/) (2021)
- [Topic Scene Graph Generation by Attention Distillation from Caption](https://mlsanity.com/paper/2110.05731/) (2021)
- [Context-aware Mixture-of-Experts for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2208.07109/) (2022)
- [From Easy to Hard: Learning Curricular Shape-aware Features for Robust Panoptic Scene Graph Generation](https://mlsanity.com/paper/2407.09191/) (2024)
- [BCTR: Bidirectional Conditioning Transformer for Scene Graph Generation](https://mlsanity.com/paper/2407.18715/) (2024)
- [Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generation](https://mlsanity.com/paper/2504.12606/) (2025)

## Abstract (arXiv)

Scene graph generation models understand the scene through object and predicate recognition, but are prone to mistakes due to the challenges of perception in the wild. Perception errors often lead to nonsensical compositions in the output scene graph, which do not follow real-world rules and patterns, and can be corrected using commonsense knowledge. We propose the first method to acquire visual commonsense such as affordance and intuitive physics automatically from data, and use that to improve the robustness of scene understanding. To this end, we extend Transformer models to incorporate the structure of scene graphs, and train our Global-Local Attention Transformer on a scene graph corpus. Once trained, our model can be applied on any scene graph generation model and correct its obvious mistakes, resulting in more semantically plausible scene graphs. Through extensive experiments, we show our model learns commonsense better than any alternative, and improves the accuracy of state-of-the-art scene graph generation methods.