# Is Medieval Distant Viewing Possible? : Extending and Enriching Annotation of Legacy Image Collections using Visual Analytics

arXiv: https://arxiv.org/abs/2208.09657 · HTML: https://mlsanity.com/paper/2208.09657/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 80 references that are in the map.
- First posted on arXiv 20 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [Hierarchical Image Classification using Entailment Cone Embeddings](https://mlsanity.com/paper/2004.03459/) (2020)
- [Hierarchy-based Image Embeddings for Semantic Image Retrieval](https://mlsanity.com/paper/1809.09924/) (2018)
- [The State of the Art in Integrating Machine Learning into Visual Analytics](https://mlsanity.com/paper/1802.07954/) (2018)
- [docExtractor: An off-the-shelf historical document element extraction](https://mlsanity.com/paper/2012.08191/) (2020)
- [EXACT: A collaboration toolset for algorithm-aided annotation of images with annotation version control](https://mlsanity.com/paper/2004.14595/) (2020)
- [VASSL: A Visual Analytics Toolkit for Social Spambot Labeling](https://mlsanity.com/paper/1907.13319/) (2019)
- [Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness](https://mlsanity.com/paper/1908.02588/) (2019)
- [Image Collation: Matching illustrations in manuscripts](https://mlsanity.com/paper/2108.08109/) (2021)
- [Perceptual Visual Interactive Learning](https://mlsanity.com/paper/1810.10789/) (2018)
- [Transcribing Medieval Manuscripts for Machine Learning](https://mlsanity.com/paper/2207.07726/) (2022)

## Built on by (most cited first, 1 of 1)

- [Visual Navigation of Digital Libraries: Retrieval and Classification of Images in the National Library of Norway's Digitised Book Collection](https://mlsanity.com/paper/2410.14969/) (2024)

## Abstract (arXiv)

Distant viewing approaches have typically used image datasets close to the contemporary image data used to train machine learning models. To work with images from other historical periods requires expert annotated data, and the quality of labels is crucial for the quality of results. Especially when working with cultural heritage collections that contain myriad uncertainties, annotating data, or re-annotating, legacy data is an arduous task. In this paper, we describe working with two pre-annotated sets of medieval manuscript images that exhibit conflicting and overlapping metadata. Since a manual reconciliation of the two legacy ontologies would be very expensive, we aim (1) to create a more uniform set of descriptive labels to serve as a "bridge" in the combined dataset, and (2) to establish a high quality hierarchical classification that can be used as a valuable input for subsequent supervised machine learning. To achieve these goals, we developed visualization and interaction mechanisms, enabling medievalists to combine, regularize and extend the vocabulary used to describe these, and other cognate, image datasets. The visual interfaces provide experts an overview of relationships in the data going beyond the sum total of the metadata. Word and image embeddings as well as co-occurrences of labels across the datasets, enable batch re-annotation of images, recommendation of label candidates and support composing a hierarchical classification of labels.