# Healthsheet: Development of a Transparency Artifact for Health Datasets

arXiv: https://arxiv.org/abs/2202.13028 · HTML: https://mlsanity.com/paper/2202.13028/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 99 references that are in the map.
- First posted on arXiv 26 Feb 2022.

## Builds on (most cited first, 16 of 16)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020)
- [Principles alone cannot guarantee ethical AI](https://mlsanity.com/paper/1906.06668/) (2019)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?](https://mlsanity.com/paper/1912.08320/) (2019)
- [Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus](https://mlsanity.com/paper/2105.05241/) (2021)
- [Deep Cox Mixtures for Survival Regression](https://mlsanity.com/paper/2101.06536/) (2021)
- [Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks](https://mlsanity.com/paper/1908.00690/) (2019)
- [Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics](https://mlsanity.com/paper/2005.07572/) (2020)
- [Towards Standardization of Data Licenses: The Montreal Data License](https://mlsanity.com/paper/1903.12262/) (2019)

## Built on by (most cited first, 23 of 23)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Learning from data with structured missingness](https://mlsanity.com/paper/2304.01429/) (2023)
- [Ethical Considerations for Responsible Data Curation](https://mlsanity.com/paper/2302.03629/) (2023)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMs](https://mlsanity.com/paper/2312.11803/) (2023)
- [The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review](https://mlsanity.com/paper/2402.13635/) (2024)
- [A domain-specific language for describing machine learning datasets](https://mlsanity.com/paper/2207.02848/) (2022)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review](https://mlsanity.com/paper/2501.10727/) (2025)
- [Augmenting Chest X-ray Datasets with Non-Expert Annotations](https://mlsanity.com/paper/2309.02244/) (2023)
- [MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs](https://mlsanity.com/paper/2505.19800/) (2025)
- [Scorecards for Synthetic Medical Data Evaluation and Reporting](https://mlsanity.com/paper/2406.11143/) (2024)
- [FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines](https://mlsanity.com/paper/2307.15475/) (2023)
- [Promoting the Responsible Development of Speech Datasets for Mental Health and Neurological Disorders Research](https://mlsanity.com/paper/2406.04116/) (2024)
- [Machine Learning and Public Health: Identifying and Mitigating Algorithmic Bias through a Systematic Review](https://mlsanity.com/paper/2510.14669/) (2025)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Development and Validation of ML-DQA -- a Machine Learning Data Quality Assurance Framework for Healthcare](https://mlsanity.com/paper/2208.02670/) (2022)
- [Towards Transparency: Exploring LLM Trainings Datasets through Visual Topic Modeling and Semantic Frame](https://mlsanity.com/paper/2406.06574/) (2024)
- [From Reflection to Repair: A Scoping Review of Dataset Documentation Tools](https://mlsanity.com/paper/2602.15968/) (2026)
- [On the Readiness of Scientific Data for a Fair and Transparent Use in Machine Learning](https://mlsanity.com/paper/2401.10304/) (2024)
- [A case for data valuation transparency via DValCards](https://mlsanity.com/paper/2506.23349/) (2025)
- [Standards in the Preparation of Biomedical Research Metadata: A Bridge2AI Perspective](https://mlsanity.com/paper/2509.10432/) (2025)
- [RCTs for Frontier AI Governance: Methodological Challenges and Solutions for Human Uplift Studies](https://mlsanity.com/paper/2603.11001/) (2026)

## Abstract (arXiv)

Machine learning (ML) approaches have demonstrated promising results in a wide range of healthcare applications. Data plays a crucial role in developing ML-based healthcare systems that directly affect people's lives. Many of the ethical issues surrounding the use of ML in healthcare stem from structural inequalities underlying the way we collect, use, and handle data. Developing guidelines to improve documentation practices regarding the creation, use, and maintenance of ML healthcare datasets is therefore of critical importance. In this work, we introduce Healthsheet, a contextualized adaptation of the original datasheet questionnaire ~citegebru2018datasheets for health-specific applications. Through a series of semi-structured interviews, we adapt the datasheets for healthcare data documentation. As part of the Healthsheet development process and to understand the obstacles researchers face in creating datasheets, we worked with three publicly-available healthcare datasets as our case studies, each with different types of structured data: Electronic health Records (EHR), clinical trial study data, and smartphone-based performance outcome measures. Our findings from the interviewee study and case studies show 1) that datasheets should be contextualized for healthcare, 2) that despite incentives to adopt accountability practices such as datasheets, there is a lack of consistency in the broader use of these practices 3) how the ML for health community views datasheets and particularly textitHealthsheets as diagnostic tool to surface the limitations and strength of datasets and 4) the relative importance of different fields in the datasheet to healthcare concerns.