# A Comprehensive Survey and Classification of Evaluation Criteria for Trustworthy Artificial Intelligence

arXiv: https://arxiv.org/abs/2410.17281 · HTML: https://mlsanity.com/paper/2410.17281/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 96 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 30 of 35)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning](https://mlsanity.com/paper/2007.09339/) (2020)
- [The Foundation Model Transparency Index](https://mlsanity.com/paper/2310.12941/) (2023)
- [Fairness in Credit Scoring: Assessment, Implementation and Profit Implications](https://mlsanity.com/paper/2103.01907/) (2021)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification](https://mlsanity.com/paper/2011.06485/) (2020)
- [Measuring Fairness in Generative Models](https://mlsanity.com/paper/2107.07754/) (2021)
- [Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling](https://mlsanity.com/paper/2402.17861/) (2024)
- [Active Fairness Auditing](https://mlsanity.com/paper/2206.08450/) (2022)
- [The Role of Explainability in Assuring Safety of Machine Learning in Healthcare](https://mlsanity.com/paper/2109.00520/) (2021)
- [Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination](https://mlsanity.com/paper/2009.12040/) (2020)
- [Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning](https://mlsanity.com/paper/2202.09027/) (2022)
- [Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP](https://mlsanity.com/paper/2305.01633/) (2023)
- [Improving Fairness and Privacy in Selection Problems](https://mlsanity.com/paper/2012.03812/) (2020)
- [A Maximal Correlation Approach to Imposing Fairness in Machine Learning](https://mlsanity.com/paper/2012.15259/) (2020)
- [Privacy and Trust Redefined in Federated Machine Learning](https://mlsanity.com/paper/2103.15753/) (2021)
- [A Transparency Index Framework for AI in Education](https://mlsanity.com/paper/2206.03220/) (2022)
- [AI Cards: Towards an Applied Framework for Machine-Readable AI and Risk Documentation Inspired by the EU AI Act](https://mlsanity.com/paper/2406.18211/) (2024)
- [Artificial Intelligence Governance for Businesses](https://mlsanity.com/paper/2011.10672/) (2020)
- [A Distributed Trust Framework for Privacy-Preserving Machine Learning](https://mlsanity.com/paper/2006.02456/) (2020)
- [Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems](https://mlsanity.com/paper/2106.03775/) (2021)
- [ESR: Ethics and Society Review of Artificial Intelligence Research](https://mlsanity.com/paper/2106.11521/) (2021)
- [Towards Standardizing AI Bias Exploration](https://mlsanity.com/paper/2405.19022/) (2024)

## Built on by (most cited first, 3 of 3)

- [Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliance](https://mlsanity.com/paper/2502.04695/) (2025)
- [AI Transparency: Governance Compliance or Stakeholder Requirements?](https://mlsanity.com/paper/2606.30652/) (2026)
- [Can Valence Reflect Morality in Natural Language? A Preliminary Annotation Study](https://mlsanity.com/paper/2607.20461/) (2026)

## Abstract (arXiv)

This paper presents a systematic review of the literature on evaluation criteria for Trustworthy Artificial Intelligence (TAI), with a focus on the seven EU principles of TAI. This systematic literature review identifies and analyses current evaluation criteria, maps them to the EU TAI principles and proposes a new classification system for each principle. The findings reveal both a need for and significant barriers to standardising criteria for TAI evaluation. The proposed classification contributes to the development, selection and standardization of evaluation criteria for TAI governance.