# AutoODD: Agentic Audits via Bayesian Red Teaming in Black-Box Models

arXiv: https://arxiv.org/abs/2509.08638 · HTML: https://mlsanity.com/paper/2509.08638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 10 of 10)

- [ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles](https://mlsanity.com/paper/2405.14062/) (2024)
- [Active Learning of Linear Embeddings for Gaussian Processes](https://mlsanity.com/paper/1310.6740/) (2013)
- [Query-Efficient Black-Box Red Teaming via Bayesian Optimization](https://mlsanity.com/paper/2305.17444/) (2023)
- [Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles](https://mlsanity.com/paper/2409.06450/) (2024)
- [AirTrack: Onboard Deep Learning Framework for Long-Range Aircraft Detection and Tracking](https://mlsanity.com/paper/2209.12849/) (2022)
- [TartanAviation: Image, Speech, and ADS-B Trajectory Datasets for Terminal Airspace Operations](https://mlsanity.com/paper/2403.03372/) (2024)
- [Bayesian Safety Validation for Failure Probability Estimation of Black-Box Systems](https://mlsanity.com/paper/2305.02449/) (2023)
- [Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach](https://mlsanity.com/paper/2412.06684/) (2024)
- [A Multi-output Gaussian Process Regression with Negative Transfer Mitigation for Generating Boundary Test Scenarios of Multi-UAV Systems](https://mlsanity.com/paper/2505.22331/) (2025)
- [Test Automation for Interactive Scenarios via Promptable Traffic Simulation](https://mlsanity.com/paper/2506.01199/) (2025)

## Abstract (arXiv)

Specialized machine learning models, regardless of architecture and training, are susceptible to failures in deployment. With their increasing use in high risk situations, the ability to audit these models by determining their operational design domain (ODD) is crucial in ensuring safety and compliance. However, given the high-dimensional input spaces, this process often requires significant human resources and domain expertise. To alleviate this, we introduce coolname, an LLM-Agent centric framework for automated generation of semantically relevant test cases to search for failure modes in specialized black-box models. By leveraging LLM-Agents as tool orchestrators, we aim to fit a uncertainty-aware failure distribution model on a learned text-embedding manifold by projecting the high-dimension input space to low-dimension text-embedding latent space. The LLM-Agent is tasked with iteratively building the failure landscape by leveraging tools for generating test-cases to probe the model-under-test (MUT) and recording the response. The agent also guides the search using tools to probe uncertainty estimate on the low dimensional manifold. We demonstrate this process in a simple case using models trained with missing digits on the MNIST dataset and in the real world setting of vision-based intruder detection for aerial vehicles.