Measuring AI Systems Beyond Accuracy
Not on the map. Why?
Abstract (arXiv)
Current test and evaluation (T&E) methods for assessing machine learning (ML) system performance often rely on incomplete metrics. Testing is additionally often siloed from the other phases of the ML system lifecycle. Research investigating cross-domain approaches to ML T&E is needed to drive the state of the art forward and to build an Artificial Intelligence (AI) engineering discipline. This paper advocates for a robust, integrated approach to testing by outlining six key questions for guiding a holistic T&E strategy.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 39 references that are in the map. First posted on arXiv 7 Apr 2022.
Builds on
6 of 39 references.
References · 6
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 414
- 2021Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systemscited by 19
- 2021On managing vulnerabilities in AI/ML systemscited by 5
Built on by
Not cited yet.