# Compensating Supervision Incompleteness with Prior Knowledge in Semantic Image Interpretation

arXiv: https://arxiv.org/abs/1910.00462 · HTML: https://mlsanity.com/paper/1910.00462/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use case](https://mlsanity.com/paper/2104.11914/) (2021)
- [Faster-LTN: a neuro-symbolic, end-to-end object detection architecture](https://mlsanity.com/paper/2107.01877/) (2021)
- [OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics](https://mlsanity.com/paper/2202.10201/) (2022)
- [Relational Graph Convolutional Networks: A Closer Look](https://mlsanity.com/paper/2107.10015/) (2021)
- [EXPLAIN, AGREE, LEARN: Scaling Learning for Neural Probabilistic Logic](https://mlsanity.com/paper/2408.08133/) (2024)
- [An Insect-Inspired Randomly, Weighted Neural Network with Random Fourier Features For Neuro-Symbolic Relational Learning](https://mlsanity.com/paper/2109.06663/) (2021)
- [Representing Prior Knowledge Using Randomly, Weighted Feature Networks for Visual Relationship Detection](https://mlsanity.com/paper/2111.10686/) (2021)
- [PROTOtypical Logic Tensor Networks (PROTO-LTN) for Zero Shot Learning](https://mlsanity.com/paper/2207.00433/) (2022)
- [Fuzzy Logic Visual Network (FLVN): A neuro-symbolic approach for visual features matching](https://mlsanity.com/paper/2307.16019/) (2023)
- [sLTN: Structural Logic Tensor Networks](https://mlsanity.com/paper/2608.11136/) (2026)

## Abstract (arXiv)

Semantic Image Interpretation is the task of extracting a structured semantic description from images. This requires the detection of visual relationships: triples (subject,relation,object) describing a semantic relation between a subject and an object. A pure supervised approach to visual relationship detection requires a complete and balanced training set for all the possible combinations of (subject, relation, object). However, such training sets are not available and would require a prohibitive human effort. This implies the ability of predicting triples which do not appear in the training set. This problem is called zero-shot learning. State-of-the-art approaches to zero-shot learning exploit similarities among relationships in the training set or external linguistic knowledge. In this paper, we perform zero-shot learning by using Logic Tensor Networks, a novel Statistical Relational Learning framework that exploits both the similarities with other seen relationships and background knowledge, expressed with logical constraints between subjects, relations and objects. The experiments on the Visual Relationship Dataset show that the use of logical constraints outperforms the current methods. This implies that background knowledge can be used to alleviate the incompleteness of training sets.