# OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

arXiv: https://arxiv.org/abs/2202.10201 · HTML: https://mlsanity.com/paper/2202.10201/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 21 Feb 2022.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [CSPNet: A New Backbone that can Enhance Learning Capability of CNN](https://mlsanity.com/paper/1911.11929/) (2019)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Learning to Compose Dynamic Tree Structures for Visual Contexts](https://mlsanity.com/paper/1812.01880/) (2018)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [RoboTHOR: An Open Simulation-to-Real Embodied AI Platform](https://mlsanity.com/paper/2004.06799/) (2020)
- [Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation](https://mlsanity.com/paper/1707.09423/) (2017)
- [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021)
- [EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use case](https://mlsanity.com/paper/2104.11914/) (2021)
- [Classification by Attention: Scene Graph Classification with Prior Knowledge](https://mlsanity.com/paper/2011.10084/) (2020)
- [Relation Transformer Network](https://mlsanity.com/paper/2004.06193/) (2020)
- [Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failures](https://mlsanity.com/paper/2108.03554/) (2021)
- [Compensating Supervision Incompleteness with Prior Knowledge in Semantic Image Interpretation](https://mlsanity.com/paper/1910.00462/) (2019)
- [Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning models](https://mlsanity.com/paper/2003.11743/) (2020)
- [RoboSherlock: Cognition-enabled Robot Perception for Everyday Manipulation Tasks](https://mlsanity.com/paper/1911.10079/) (2019)

## Built on by (most cited first, 9 of 9)

- [Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge](https://mlsanity.com/paper/2311.12889/) (2023)
- [Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generation](https://mlsanity.com/paper/2305.18668/) (2023)
- [GraSP-VLA: Graph-based Symbolic Action Representation for Long-Horizon Planning with VLA Policies](https://mlsanity.com/paper/2511.04357/) (2025)
- [FORESCENE: FOREcasting human activity via latent SCENE graphs diffusion](https://mlsanity.com/paper/2503.06182/) (2025)
- [Measuring Image-Relation Alignment: Reference-Free Evaluation of VLMs and Synthetic Pre-training for Open-Vocabulary Scene Graph Generation](https://mlsanity.com/paper/2509.01209/) (2025)
- [REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation](https://mlsanity.com/paper/2603.06386/) (2026)
- [STDG: Semi-Teacher-Student Training Paradigram for Depth-guided One-stage Scene Graph Generation](https://mlsanity.com/paper/2309.08179/) (2023)
- [A transition towards virtual representations of visual scenes](https://mlsanity.com/paper/2410.07987/) (2024)
- [Visual Commonsense Driven Knowledge Refinements for Scene Graph Generation](https://mlsanity.com/paper/2606.06369/) (2026)

## Abstract (arXiv)

Scene graph generation from images is a task of great interest to applications such as robotics, because graphs are the main way to represent knowledge about the world and regulate human-robot interactions in tasks such as Visual Question Answering (VQA). Unfortunately, its corresponding area of machine learning is still relatively in its infancy, and the solutions currently offered do not specialize well in concrete usage scenarios. Specifically, they do not take existing "expert" knowledge about the domain world into account; and that might indeed be necessary in order to provide the level of reliability demanded by the use case scenarios. In this paper, we propose an initial approximation to a framework called Ontology-Guided Scene Graph Generation (OG-SGG), that can improve the performance of an existing machine learning based scene graph generator using prior knowledge supplied in the form of an ontology (specifically, using the axioms defined within); and we present results evaluated on a specific scenario founded in telepresence robotics. These results show quantitative and qualitative improvements in the generated scene graphs.