# Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards arXiv: https://arxiv.org/abs/2108.07374 ยท HTML: https://mlsanity.com/paper/2108.07374/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 15 of its 38 references that are in the map. - First posted on arXiv 16 Aug 2021. ## Builds on (most cited first, 15 of 15) - [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015) - [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018) - [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018) - [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019) - [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020) - [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020) - [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://mlsanity.com/paper/2102.01672/) (2021) - [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019) - [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020) - [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020) - [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018) - [Disembodied Machine Learning: On the Illusion of Objectivity in NLP](https://mlsanity.com/paper/2101.11974/) (2021) - [It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia](https://mlsanity.com/paper/1501.06307/) (2015) - [The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLP](https://mlsanity.com/paper/2103.09710/) (2021) - [DataHub: Collaborative Data Science & Dataset Version Management at Scale](https://mlsanity.com/paper/1409.0798/) (2014) ## Built on by (most cited first, 24 of 24) - [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021) - [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022) - [Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators](https://mlsanity.com/paper/2307.05532/) (2023) - [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022) - [GEMv2: Multilingual NLG Benchmarking in a Single Line of Code](https://mlsanity.com/paper/2206.11249/) (2022) - [Data Governance in the Age of Large-Scale Data-Driven Language Technology](https://mlsanity.com/paper/2206.03216/) (2022) - [OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objects](https://mlsanity.com/paper/2309.07921/) (2023) - [Where to start? Analyzing the potential value of intermediate models](https://mlsanity.com/paper/2211.00107/) (2022) - [SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets](https://mlsanity.com/paper/2111.06467/) (2021) - [SLING: Sino Linguistic Evaluation of Large Language Models](https://mlsanity.com/paper/2210.11689/) (2022) - [A domain-specific language for describing machine learning datasets](https://mlsanity.com/paper/2207.02848/) (2022) - [Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trends](https://mlsanity.com/paper/2401.13177/) (2024) - [Saliency Cards: A Framework to Characterize and Compare Saliency Methods](https://mlsanity.com/paper/2206.02958/) (2022) - [What's documented in AI? Systematic Analysis of 32K AI Model Cards](https://mlsanity.com/paper/2402.05160/) (2024) - [Seeing without Looking: Analysis Pipeline for Child Sexual Abuse Datasets](https://mlsanity.com/paper/2204.14110/) (2022) - [Open Datasheets: Machine-readable Documentation for Open Datasets and Responsible AI Assessments](https://mlsanity.com/paper/2312.06153/) (2023) - [The State of Documentation Practices of Third-party Machine Learning Models and Datasets](https://mlsanity.com/paper/2312.15058/) (2023) - [OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning](https://mlsanity.com/paper/2403.05565/) (2024) - [Using Large Language Models to Enrich the Documentation of Datasets for Machine Learning](https://mlsanity.com/paper/2404.15320/) (2024) - [Optimizing Image Capture for Computer Vision-Powered Taxonomic Identification and Trait Recognition of Biodiversity Specimens](https://mlsanity.com/paper/2505.17317/) (2025) - [Modeling the Sacred: Considerations when Using Religious Texts in Natural Language Processing](https://mlsanity.com/paper/2404.14740/) (2024) - [From Reflection to Repair: A Scoping Review of Dataset Documentation Tools](https://mlsanity.com/paper/2602.15968/) (2026) - [On the Readiness of Scientific Data for a Fair and Transparent Use in Machine Learning](https://mlsanity.com/paper/2401.10304/) (2024) - [MRM3: Machine Readable ML Model Metadata](https://mlsanity.com/paper/2505.13343/) (2025) ## Abstract (arXiv) Developing documentation guidelines and easy-to-use templates for datasets and models is a challenging task, especially given the variety of backgrounds, skills, and incentives of the people involved in the building of natural language processing (NLP) tools. Nevertheless, the adoption of standard documentation practices across the field of NLP promotes more accessible and detailed descriptions of NLP datasets and models, while supporting researchers and developers in reflecting on their work. To help with the standardization of documentation, we present two case studies of efforts that aim to develop reusable documentation templates -- the HuggingFace data card, a general purpose card for datasets in NLP, and the GEM benchmark data and model cards with a focus on natural language generation. We describe our process for developing these templates, including the identification of relevant stakeholder groups, the definition of a set of guiding principles, the use of existing templates as our foundation, and iterative revisions based on feedback.