# NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

arXiv: https://arxiv.org/abs/2606.20408 · HTML: https://mlsanity.com/paper/2606.20408/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 15 of 15)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024)
- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [Red-Teaming LLM Multi-Agent Systems via Communication Attacks](https://mlsanity.com/paper/2502.14847/) (2025)
- [OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://mlsanity.com/paper/2506.14866/) (2025)
- [OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety](https://mlsanity.com/paper/2507.06134/) (2025)
- [Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles](https://mlsanity.com/paper/2408.04686/) (2024)
- [BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks](https://mlsanity.com/paper/2508.08127/) (2025)
- [Multi-Turn Jailbreaks Are Simpler Than They Seem](https://mlsanity.com/paper/2508.07646/) (2025)
- [SECURE: Benchmarking Large Language Models for Cybersecurity](https://mlsanity.com/paper/2405.20441/) (2024)
- [MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety](https://mlsanity.com/paper/2605.01687/) (2026)
- [CritBench: A Framework for Evaluating Cybersecurity Capabilities of Large Language Models in IEC 61850 Digital Substation Environments](https://mlsanity.com/paper/2604.06019/) (2026)

## Built on by (most cited first, 2 of 2)

- [Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems](https://mlsanity.com/paper/2609.17320/) (2026)
- [A Governance-Aware Large Language Model Orchestrated Agentic Digital Twin for Transmission System Operator Control Room Decision Support](https://mlsanity.com/paper/2609.22476/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized. We present NRT-Bench, a benchmark for multi-turn red-teaming of LLM agents acting as operators of a safety-critical system, instantiated in a simulated nuclear power plant control room. A five-role operator team, each backed by a configurable LLM, runs a plant governed by six critical safety functions (CSFs), while adversaries inject messages over four channels in bounded multi-turn sessions with per-turn feedback. Harm is an objective signal rather than LLM-judged text: a run terminates the moment any CSF is lost, attributed to the causing message. Evaluating four frontier operator models under a fixed-attack paired-replay protocol, we find that adaptive multi-turn attacks reliably push the operator team past a safety limit: across the four models, between 8.7% and 12.1% of attack sessions end with the plant losing a critical safety function. Although the four models look almost equally robust by this aggregate rate, their failures barely overlap: of 149 sessions, none defeat all four models while a third defeat at least one, so vulnerabilities are nearly disjoint across models rather than nested. The effect of added defences is strongly model-dependent: the same guardrail stack or safety-advisor agent that lowers attack success for one model can raise it for another. We release the simulation venue, attack dataset, and replay tooling for reproducible safety evaluation of LLM agents.