# Open-World Visual Recognition Using Knowledge Graphs

arXiv: https://arxiv.org/abs/1708.08310 · HTML: https://mlsanity.com/paper/1708.08310/

## Key facts

- Cited by 5 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 28 Aug 2017.

## Built on by (most cited first, 5 of 5)

- [Open-world Machine Learning: Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2105.13448/) (2021)
- [Revisiting Open World Object Detection](https://mlsanity.com/paper/2201.00471/) (2022)
- [A Survey on Visual Transfer Learning using Knowledge Graphs](https://mlsanity.com/paper/2201.11794/) (2022)
- [Egocentric Hierarchical Visual Semantics](https://mlsanity.com/paper/2305.05422/) (2023)
- [Learning for Transductive Threshold Calibration in Open-World Recognition](https://mlsanity.com/paper/2305.12039/) (2023)

## Abstract (arXiv)

In a real-world setting, visual recognition systems can be brought to make predictions for images belonging to previously unknown class labels. In order to make semantically meaningful predictions for such inputs, we propose a two-step approach that utilizes information from knowledge graphs. First, a knowledge-graph representation is learned to embed a large set of entities into a semantic space. Second, an image representation is learned to embed images into the same space. Under this setup, we are able to predict structured properties in the form of relationship triples for any open-world image. This is true even when a set of labels has been omitted from the training protocols of both the knowledge graph and image embeddings. Furthermore, we append this learning framework with appropriate smoothness constraints and show how prior knowledge can be incorporated into the model. Both these improvements combined increase performance for visual recognition by a factor of six compared to our baseline. Finally, we propose a new, extended dataset which we use for experiments.