# PLCBench: Can Autonomous LLM Agents Turn PLC Access into Sustained Physical Impact?

arXiv: https://arxiv.org/abs/2608.26882 · HTML: https://mlsanity.com/paper/2608.26882/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 74 references that are in the map.
- First posted on arXiv 27 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models](https://mlsanity.com/paper/2408.08926/) (2024)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agents](https://mlsanity.com/paper/2410.14209/) (2024)
- [ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents](https://mlsanity.com/paper/2605.14153/) (2026)
- [Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges](https://mlsanity.com/paper/2506.17644/) (2025)
- [Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios](https://mlsanity.com/paper/2603.11214/) (2026)
- [ICSREF: A Framework for Automated Reverse Engineering of Industrial Control Systems Binaries](https://mlsanity.com/paper/1812.03478/) (2018)
- [AttackLLM: LLM-based Attack Pattern Generation for an Industrial Control System](https://mlsanity.com/paper/2504.04187/) (2025)
- [SCAPHY: Detecting Modern ICS Attacks by Correlating Behaviors in SCADA and PHYsical](https://mlsanity.com/paper/2211.14642/) (2022)
- [ICSPatch: Automated Vulnerability Localization and Non-Intrusive Hotpatching in Industrial Control Systems using Data Dependence Graphs](https://mlsanity.com/paper/2212.04229/) (2022)
- [SoK: Security of Programmable Logic Controllers](https://mlsanity.com/paper/2403.00280/) (2024)

## Abstract (arXiv)

Industrial control systems (ICSs) rely on programmable logic controllers (PLCs) to connect networked computation with physical control. Tool-using large language model (LLM) agents represent an emerging attack threat: can an autonomous agent convert a network-reachable PLC into sustained adverse physical impact? However, existing evaluations focus on digital tasks or individual stages of PLC testing. In ICSs, evaluations that stop at software exploitation, an accepted write, or tool access may therefore mischaracterize physical risk. We present PLCBENCH, to our knowledge, the first real-PLC hardware-in-the-loop (HIL) framework for characterizing this cyber-to-physical capability and its boundaries. It combines vendor-native interaction, commercial PLC execution, closed-loop reduced-order process simulation, and independent outcome verification. A deterministic evaluator applies fixed rules to runner, communication, PLC-object, and process records to assign six hidden diagnostic flags, distinguishing usable PLC interaction, process-linked manipulation, and sustained physical impact. We instantiate PLCBENCH on four commercial PLCs crossed with four closed-loop workloads. Across five LLM families and 240 real-PLC episodes, 75 episodes (31.3%) sustain their respective physical objectives. Stagewise results show that 98 episodes stop before a valid native read, whereas 62 reach a process-linked write but do not sustain the final objective. Notably, richer process observation is associated with an increase in conditional objective attainment after a process-linked write from 44.2% to 64.0%. These measurements localize failure in configured PLC-process deployments and identify intervention points for future defense evaluation. To support reproducibility, we release the safely disclosable PLCBENCH code and a software-only reproduction pipeline through the accompanying artifact.