# Certification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study

arXiv: https://arxiv.org/abs/2305.18307 · HTML: https://mlsanity.com/paper/2305.18307/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 83 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 21 of 21)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [Artificial Intelligence: the global landscape of ethics guidelines](https://mlsanity.com/paper/1906.11668/) (2019)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [The relationship between trust in AI and trustworthy machine learning technologies](https://mlsanity.com/paper/1912.00782/) (2019)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [How Different Groups Prioritize Ethical Values for Responsible AI](https://mlsanity.com/paper/2205.07722/) (2022)
- [Designing for Responsible Trust in AI Systems: A Communication Perspective](https://mlsanity.com/paper/2204.13828/) (2022)
- [Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences](https://mlsanity.com/paper/2001.09604/) (2020)
- [Algorithmic Auditing and Social Justice: Lessons from the History of Audit Studies](https://mlsanity.com/paper/2109.06974/) (2021)
- [Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits](https://mlsanity.com/paper/2102.04256/) (2021)
- [The Sanction of Authority: Promoting Public Trust in AI](https://mlsanity.com/paper/2102.04221/) (2021)
- [Filling gaps in trustworthy development of AI](https://mlsanity.com/paper/2112.07773/) (2021)
- [Towards a multi-stakeholder value-based assessment framework for algorithmic systems](https://mlsanity.com/paper/2205.04525/) (2022)
- [Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measures](https://mlsanity.com/paper/2203.12318/) (2022)

## Built on by (most cited first, 5 of 5)

- [Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articles](https://mlsanity.com/paper/2510.21293/) (2025)
- [Surveys Considered Harmful? Reflecting on the Use of Surveys in AI Research, Development, and Governance](https://mlsanity.com/paper/2408.01458/) (2024)
- [More Human or More AI? Visualizing Human-AI Collaboration Disclosures in Journalistic News Production](https://mlsanity.com/paper/2601.11072/) (2026)
- [Bridging the Communication Gap: Evaluating AI Labeling Practices for Trustworthy AI Development](https://mlsanity.com/paper/2501.11909/) (2025)
- [A Critical Analysis of Trustworthy AI Tools, Mark Frameworks, and the Implementation Chasms](https://mlsanity.com/paper/2607.15480/) (2026)

## Abstract (arXiv)

Auditing plays a pivotal role in the development of trustworthy AI. However, current research primarily focuses on creating auditable AI documentation, which is intended for regulators and experts rather than end-users affected by AI decisions. How to communicate to members of the public that an AI has been audited and considered trustworthy remains an open challenge. This study empirically investigated certification labels as a promising solution. Through interviews (N = 12) and a census-representative survey (N = 302), we investigated end-users' attitudes toward certification labels and their effectiveness in communicating trustworthiness in low- and high-stakes AI scenarios. Based on the survey results, we demonstrate that labels can significantly increase end-users' trust and willingness to use AI in both low- and high-stakes scenarios. However, end-users' preferences for certification labels and their effect on trust and willingness to use AI were more pronounced in high-stake scenarios. Qualitative content analysis of the interviews revealed opportunities and limitations of certification labels, as well as facilitators and inhibitors for the effective use of labels in the context of AI. For example, while certification labels can mitigate data-related concerns expressed by end-users (e.g., privacy and data protection), other concerns (e.g., model performance) are more challenging to address. Our study provides valuable insights and recommendations for designing and implementing certification labels as a promising constituent within the trustworthy AI ecosystem.