# Classification by Attention: Scene Graph Classification with Prior Knowledge

arXiv: https://arxiv.org/abs/2011.10084 · HTML: https://mlsanity.com/paper/2011.10084/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 26 of 26)

- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Recovering the Unbiased Scene Graphs from the Biased Ones](https://mlsanity.com/paper/2107.02112/) (2021)
- [Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2203.09811/) (2022)
- [Relationformer: A Unified Framework for Image-to-Graph Generation](https://mlsanity.com/paper/2203.10202/) (2022)
- [VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloud](https://mlsanity.com/paper/2303.14408/) (2023)
- [EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use case](https://mlsanity.com/paper/2104.11914/) (2021)
- [Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning](https://mlsanity.com/paper/2303.08046/) (2023)
- [SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstruction](https://mlsanity.com/paper/2309.15702/) (2023)
- [Synth2: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings](https://mlsanity.com/paper/2403.07750/) (2024)
- [Improving Scene Graph Classification by Exploiting Knowledge from Texts](https://mlsanity.com/paper/2102.04760/) (2021)
- [OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics](https://mlsanity.com/paper/2202.10201/) (2022)
- [The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding](https://mlsanity.com/paper/2109.13392/) (2021)
- [Probabilistic Debiasing of Scene Graphs](https://mlsanity.com/paper/2211.06444/) (2022)
- [Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining](https://mlsanity.com/paper/2305.14281/) (2023)
- [MemeGraphs: Linking Memes to Knowledge Graphs](https://mlsanity.com/paper/2305.18391/) (2023)
- [Weakly-Supervised 3D Scene Graph Generation via Visual-Linguistic Assisted Pseudo-labeling](https://mlsanity.com/paper/2404.02527/) (2024)
- [AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generation](https://mlsanity.com/paper/2404.07788/) (2024)
- [Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Images](https://mlsanity.com/paper/2508.06546/) (2025)
- [Explore Contextual Information for 3D Scene Graph Generation](https://mlsanity.com/paper/2210.06240/) (2022)
- [Object-Centric Representation Learning for Enhanced 3D Semantic Scene Graph Prediction](https://mlsanity.com/paper/2510.04714/) (2025)
- [Prior-RadGraphFormer: A Prior-Knowledge-Enhanced Transformer for Generating Radiology Graphs from X-Rays](https://mlsanity.com/paper/2303.13818/) (2023)
- [Edge-Centric Relational Reasoning for 3D Scene Graph Prediction](https://mlsanity.com/paper/2511.15288/) (2025)
- [Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation](https://mlsanity.com/paper/2203.09160/) (2022)
- [Relational Prior Knowledge Graphs for Detection and Instance Segmentation](https://mlsanity.com/paper/2310.07573/) (2023)
- [Fine-Grained Scene Graph Generation via Sample-Level Bias Prediction](https://mlsanity.com/paper/2407.19259/) (2024)
- [SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation](https://mlsanity.com/paper/2606.15328/) (2026)

## Abstract (arXiv)

A major challenge in scene graph classification is that the appearance of objects and relations can be significantly different from one image to another. Previous works have addressed this by relational reasoning over all objects in an image or incorporating prior knowledge into classification. Unlike previous works, we do not consider separate models for perception and prior knowledge. Instead, we take a multi-task learning approach, where we implement the classification as an attention layer. This allows for the prior knowledge to emerge and propagate within the perception model. By enforcing the model also to represent the prior, we achieve a strong inductive bias. We show that our model can accurately generate commonsense knowledge and that the iterative injection of this knowledge to scene representations leads to significantly higher classification performance. Additionally, our model can be fine-tuned on external knowledge given as triples. When combined with self-supervised learning and with 1% of annotated images only, this gives more than 3% improvement in object classification, 26% in scene graph classification, and 36% in predicate prediction accuracy.