# Multimodal Cultural Heritage Knowledge Graph Extension with Language and Vision Models

arXiv: https://arxiv.org/abs/2605.17669 · HTML: https://mlsanity.com/paper/2605.17669/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 54 references that are in the map.
- First posted on arXiv 17 May 2026.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion](https://mlsanity.com/paper/1811.04441/) (2018)
- [TorusE: Knowledge Graph Embedding on a Lie Group](https://mlsanity.com/paper/1711.05435/) (2017)
- [Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction](https://mlsanity.com/paper/1307.7973/) (2013)
- [ArCo: the Italian Cultural Heritage Knowledge Graph](https://mlsanity.com/paper/1905.02840/) (2019)
- [Evaluating Language Models for Knowledge Base Completion](https://mlsanity.com/paper/2303.11082/) (2023)
- [Contextual Graph Attention for Answering Logical Queries over Incomplete Knowledge Graphs](https://mlsanity.com/paper/1910.00084/) (2019)
- [Enhancing semantic expressivity in the cultural heritage domain: exposing the Zeri Photo Archive as Linked Open Data](https://mlsanity.com/paper/1605.01188/) (2016)
- [WikiDataSets: Standardized sub-graphs from Wikidata](https://mlsanity.com/paper/1906.04536/) (2019)
- [Refining Wikidata Taxonomy using Large Language Models](https://mlsanity.com/paper/2409.04056/) (2024)
- [Wikidata-lite for Knowledge Extraction and Exploration](https://mlsanity.com/paper/2211.05416/) (2022)

## Built on by (most cited first, 1 of 1)

- [Automated Construction of FAIR Digital Object Knowledge Graphs from Flat Cultural Heritage Records](https://mlsanity.com/paper/2608.23263/) (2026)

## Abstract (arXiv)

The preservation and interpretation of cultural heritage increasingly rely on digital technologies, among which Knowledge Graphs (KGs) stand out for their ability to structure vast amounts of data. However, the construction and expansion of these KGs often face challenges due to the diverse and complex nature of cultural heritage information. In this paper, we propose a novel approach for extending KG resources in the domain of cultural heritage, which we applied to French data. First, we introduce a new knowledge graph in the domain of French cultural heritage, WJoconde, which is distinguished by its multimodality as it integrates both textual and image information of the entities. We further introduce three variants of WJoconde to facilitate downstream research, such as Knowledge Graph Completion (KGC). We also built a comprehensive benchmark for KGC methods on our dataset. Second, we propose a new framework for extending cultural heritage KGs using multi-modal approaches leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs), which includes automated data extraction from unstructured resources combined with a special validation pipeline for grounding the output of both models, to further extend WJoconde. Our results show that by integrating the rich text and image information in cultural heritage data, we can efficiently enhance KGs with high reliability. We open-source all code and benchmark datasets with text and images, as well as the original data with an interactive access point