# Prompt me a Dataset: An investigation of text-image prompting for historical image dataset creation using foundation models

arXiv: https://arxiv.org/abs/2309.01674 · HTML: https://mlsanity.com/paper/2309.01674/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [docExtractor: An off-the-shelf historical document element extraction](https://mlsanity.com/paper/2012.08191/) (2020)
- [A Survey of Historical Document Image Datasets](https://mlsanity.com/paper/2203.08504/) (2022)
- [HORAE: an annotated dataset of books of hours](https://mlsanity.com/paper/2012.00351/) (2020)

## Built on by (most cited first, 1 of 1)

- [iDocV2: Leveraging Self-Supervision and Open-Set Detection for Improving Pattern Spotting in Historical Documents](https://mlsanity.com/paper/2604.16726/) (2026)

## Abstract (arXiv)

In this paper, we present a pipeline for image extraction from historical documents using foundation models, and evaluate text-image prompts and their effectiveness on humanities datasets of varying levels of complexity. The motivation for this approach stems from the high interest of historians in visual elements printed alongside historical texts on the one hand, and from the relative lack of well-annotated datasets within the humanities when compared to other domains. We propose a sequential approach that relies on GroundDINO and Meta's Segment-Anything-Model (SAM) to retrieve a significant portion of visual data from historical documents that can then be used for downstream development tasks and dataset creation, as well as evaluate the effect of different linguistic prompts on the resulting detections.