# Vidya: An AI-Driven Modular Pipeline for Archival Automation and Semantic Metadata Enrichment

arXiv: https://arxiv.org/abs/2605.16338 · HTML: https://mlsanity.com/paper/2605.16338/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 19 references that are in the map.
- First posted on arXiv 7 May 2026.

## Builds on (most cited first, 1 of 1)

- [Web Archives Metadata Generation with GPT-4o: Challenges and Insights](https://mlsanity.com/paper/2411.05409/) (2024)

## Abstract (arXiv)

The large-scale digitization of historical archives has created a paradox: "dark data"-digital objects lacking metadata for retrieval. Manual archival description is slow and expensive, limiting discovery and reuse. We propose Vidya, a modular pipeline that orchestrates Large Language Models (LLMs) and FOSS tools to automate semantic enrichment and archival ingestion at scale. Vidya constrains generations using YAML-defined ontologies and Pydantic validation, producing deterministic, structured JSON outputs from probabilistic models. Developed at Laboratory for Digital Humanities and Innovation (LAMUHDI) of the State University of Ponta Grossa (UEPG), Vidya applies Maker principles and open-source practices to enable low-cost deployment in memory institutions using modest hardware. We compare LLM performance and present a cost-benefit analysis showing major gains, reducing processing time from decades to days while complying with NOBRADE and ISAD(G).