# The "Collections as ML Data" Checklist for Machine Learning & Cultural Heritage

arXiv: https://arxiv.org/abs/2207.02960 · HTML: https://mlsanity.com/paper/2207.02960/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 91 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI](https://mlsanity.com/paper/2204.01075/) (2022)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk](https://mlsanity.com/paper/1712.05796/) (2017)
- [FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity](https://mlsanity.com/paper/1808.07261/) (2018)
- [System Cards for AI-Based Decision-Making for Public Policy](https://mlsanity.com/paper/2203.04754/) (2022)
- [An Algorithmic Equity Toolkit for Technology Audits by Community Advocates and Activists](https://mlsanity.com/paper/1912.02943/) (2019)
- [Comprehensive and Comprehensible Data Catalogs: The What, Who, Where, When, Why, and How of Metadata Management](https://mlsanity.com/paper/2103.07532/) (2021)

## Abstract (arXiv)

Within the cultural heritage sector, there has been a growing and concerted effort to consider a critical sociotechnical lens when applying machine learning techniques to digital collections. Though the cultural heritage community has collectively developed an emerging body of work detailing responsible operations for machine learning in libraries and other cultural heritage institutions at the organizational level, there remains a paucity of guidelines created specifically for practitioners embarking on machine learning projects. The manifold stakes and sensitivities involved in applying machine learning to cultural heritage underscore the importance of developing such guidelines. This paper contributes to this need by formulating a detailed checklist with guiding questions and practices that can be employed while developing a machine learning project that utilizes cultural heritage data. I call the resulting checklist the "Collections as ML Data" checklist, which, when completed, can be published with the deliverables of the project. By surveying existing projects, including my own project, Newspaper Navigator, I justify the "Collections as ML Data" checklist and demonstrate how the formulated guiding questions can be employed and operationalized.