# Automated Goldsmith's Mark Retrieval in Silverware

arXiv: https://arxiv.org/abs/2609.20509 · HTML: https://mlsanity.com/paper/2609.20509/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 17 Sep 2026.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [A Decade Survey of Content Based Image Retrieval using Deep Learning](https://mlsanity.com/paper/2012.00641/) (2020)
- [PyTorch Metric Learning](https://mlsanity.com/paper/2008.09164/) (2020)
- [Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models](https://mlsanity.com/paper/2606.03748/) (2026)
- [Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval](https://mlsanity.com/paper/2104.04991/) (2021)
- [iART: A Search Engine for Art-Historical Images to Support Research in the Humanities](https://mlsanity.com/paper/2108.01542/) (2021)
- [Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History](https://mlsanity.com/paper/2301.05124/) (2023)
- [Novel Artistic Scene-Centric Datasets for Effective Transfer Learning in Fragrant Spaces](https://mlsanity.com/paper/2407.11701/) (2024)
- [Smelly, dense, and spreaded: The Object Detection for Olfactory References (ODOR) dataset](https://mlsanity.com/paper/2507.08384/) (2025)
- [Iconographic Classification and Content-Based Recommendation for Digitized Artworks](https://mlsanity.com/paper/2602.19698/) (2026)

## Abstract (arXiv)

For art historians, goldsmith marks play a critical role in the identification and dating of artifacts. In practice, experts must manually compare a query mark against hundreds of documented examples, a process that is both tedious and highly dependent on specialist knowledge. To address this, we present an AI-assisted retrieval pipeline that combines mark localization with metric-learning fine-tuning across three backbone architectures: an ImageNet-pretrained ResNet-50, a supervised ViT-S/16, and a self-supervised DINOv2 ViT-S/14. We conduct a systematic evaluation of cropping strategies, where we measure the impact of no cropping, manual ground-truth cropping, and learned detection-based cropping, and assess their interaction with each backbone. Our strongest configuration, DINOv2 ViT-S/14 with manual crop and metric-learning fine-tuning, achieves an mAP of 62.63% and a Top-1 accuracy of 73.74%. Our experiments show that self-supervised pretraining and mark localization are the two most impactful factors, with learned cropping recovering the majority of the gain from manual cropping without requiring ground-truth annotations at inference time. To enable reproducibility and adoption in the digital humanities, we release our manually annotated dataset and codebase, and deploy the system via a public web interface.