# Testing Framework for Black-box AI Models

arXiv: https://arxiv.org/abs/2102.06166 · HTML: https://mlsanity.com/paper/2102.06166/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 15 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [Metamorphic Testing of a Deep Learning based Forecaster](https://mlsanity.com/paper/1907.06632/) (2019)

## Built on by (most cited first, 3 of 3)

- [pvCNN: Privacy-Preserving and Verifiable Convolutional Neural Network Testing](https://mlsanity.com/paper/2201.09186/) (2022)
- [Quantitative AI Risk Assessments: Opportunities and Challenges](https://mlsanity.com/paper/2209.06317/) (2022)
- [Automated Testing of AI Models](https://mlsanity.com/paper/2110.03320/) (2021)

## Abstract (arXiv)

With widespread adoption of AI models for important decision making, ensuring reliability of such models remains an important challenge. In this paper, we present an end-to-end generic framework for testing AI Models which performs automated test generation for different modalities such as text, tabular, and time-series data and across various properties such as accuracy, fairness, and robustness. Our tool has been used for testing industrial AI models and was very effective to uncover issues present in those models. Demo video link: https://youtu.be/984UCU17YZI