# ADMITBench: A Safety-Governed Reference Framework for Evaluating the Admissibility of Industrial LLM Advisories

arXiv: https://arxiv.org/abs/2608.03866 · HTML: https://mlsanity.com/paper/2608.03866/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 14 of 14)

- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025)
- [ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning](https://mlsanity.com/paper/2503.22738/) (2025)
- [GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning](https://mlsanity.com/paper/2406.09187/) (2024)
- [Autonomous Industrial Control using an Agentic Framework with Large Language Models](https://mlsanity.com/paper/2411.05904/) (2024)
- [AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance](https://mlsanity.com/paper/2506.03828/) (2025)
- [Leveraging LLM Agents and Digital Twins for Fault Handling in Process Plants](https://mlsanity.com/paper/2505.02076/) (2025)
- [The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS](https://mlsanity.com/paper/2102.12981/) (2021)
- [Autonomous Control Leveraging LLMs: An Agentic Framework for Next-Generation Industrial Automation](https://mlsanity.com/paper/2507.07115/) (2025)
- [PHMForge: Evaluating LLM Agents on Industrial Prognostics through MCP-Native, Algorithm-Grounded Tools](https://mlsanity.com/paper/2604.01532/) (2026)
- [PHM-Bench: A Domain-Specific Benchmarking Framework for Systematic Evaluation of Large Models in Prognostics and Health Management](https://mlsanity.com/paper/2508.02490/) (2025)
- [From Detection to Action: Using LLM Agents for Fault-Tolerant Control](https://mlsanity.com/paper/2606.28011/) (2026)

## Abstract (arXiv)

This white paper presents ADMITBench, a reference framework for evaluating industrial LLM advisories at the level of the proposed action. The framework implements a versioned, safety-governed evaluation contract that checks whether a recommendation is supported by the available evidence, permitted under the stated authority and procedure, and acceptable under the plant-specific consequence checks encoded in the selected evaluation profile. In this report, safety-governed means that eligibility is determined through explicit, non-compensatory checks derived from a versioned plant profile; it does not mean that the evaluator, model, or plant has been safety-certified. Release 0.1.0 is a public reference implementation for technical and research evaluation, not an authorisation for physical execution.