mlsanityData to 13 Oct 2026

Do-AIQ: A Design-of-Experiment Approach to Quality Evaluation of AI Mislabel Detection Algorithm

arXiv 2208.09953 ↗First posted 21 Aug 2022stat.ML cs.LG stat.APcited by 0

Not on the map. Why?

Abstract (arXiv)

The quality of Artificial Intelligence (AI) algorithms is of significant importance for confidently adopting algorithms in various applications such as cybersecurity, healthcare, and autonomous driving. This work presents a principled framework of using a design-of-experimental approach to systematically evaluate the quality of AI algorithms, named as Do-AIQ. Specifically, we focus on investigating the quality of the AI mislabel data algorithm against data poisoning. The performance of AI algorithms is affected by hyperparameters in the algorithm and data quality, particularly, data mislabeling, class imbalance, and data types. To evaluate the quality of the AI algorithms and obtain a trustworthy assessment on the quality of the algorithms, we establish a design-of-experiment framework to construct an efficient space-filling design in a high-dimensional constraint space and develop an effective surrogate model using additive Gaussian process to enable the emulation of the quality of AI algorithms. Both theoretical and numerical studies are conducted to justify the merits of the proposed framework. The proposed framework can set an exemplar for AI algorithm to enhance the AI assurance of robustness, reproducibility, and transparency.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 10 of its 37 references that are in the map. First posted on arXiv 21 Aug 2022.

Builds on

10 of 37 references.

References · 10

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,328
  2. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  3. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  4. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  5. 2018A Frank-Wolfe Framework for Efficient and Effective Adversarial Attackscited by 29
  6. 2018Robustness Guarantees for Bayesian Inference with Gaussian Processescited by 23
  7. 2015Gaussian Process Planning with Lipschitz Continuous Reward Functions: Towards Unifying Bayesian Optimization, Active Learning, and Beyondcited by 15
  8. 2022Modeling the Machine Learning Multiversecited by 11
  9. 2019Justification-Based Reliability in Machine Learningcited by 5
  10. 2021Assessing the Quality of the Datasets by Identifying Mislabeled Samplescited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation
  2. 2025Detecting and Preventing Data Poisoning Attacks on AI Models
  3. 2026Sanity Checks for Agentic Data Science
  4. 2025Adaptive Monitoring and Real-World Evaluation of Agentic AI Systems
  5. 2025TAI Scan Tool: A RAG-Based Tool With Minimalistic Input for Trustworthy AI Self-Assessment
  6. 2026MedQ-Engine: A Closed-Loop Data Engine for Evolving MLLMs in Medical Image Quality Assessment
  7. 2026Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions
  8. 2026An Agentic Retrieval Framework for Autonomous Context-Aware Data Quality Assessment