# Measuring AI Systems Beyond Accuracy

arXiv: https://arxiv.org/abs/2204.04211 · HTML: https://mlsanity.com/paper/2204.04211/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 39 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 6 of 6)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems](https://mlsanity.com/paper/2103.14101/) (2021)
- [On managing vulnerabilities in AI/ML systems](https://mlsanity.com/paper/2101.10865/) (2021)

## Abstract (arXiv)

Current test and evaluation (T&E) methods for assessing machine learning (ML) system performance often rely on incomplete metrics. Testing is additionally often siloed from the other phases of the ML system lifecycle. Research investigating cross-domain approaches to ML T&E is needed to drive the state of the art forward and to build an Artificial Intelligence (AI) engineering discipline. This paper advocates for a robust, integrated approach to testing by outlining six key questions for guiding a holistic T&E strategy.