# PLCWorld: Benchmarking LLM-Generated PLC Programs in Closed-Loop Plant Simulation

arXiv: https://arxiv.org/abs/2610.02982 · HTML: https://mlsanity.com/paper/2610.02982/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 8 of 8)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agents](https://mlsanity.com/paper/2410.14209/) (2024)
- [LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems](https://mlsanity.com/paper/2401.05443/) (2024)
- [BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Construction](https://mlsanity.com/paper/2510.16559/) (2025)
- [MCCoder: Streamlining Motion Control with LLM-Assisted Code Generation and Rigorous Verification](https://mlsanity.com/paper/2410.15154/) (2024)
- [Spec2Control: Automating PLC/DCS Control-Logic Engineering from Natural Language Requirements with LLMs - A Multi-Plant Evaluation](https://mlsanity.com/paper/2510.04519/) (2025)
- [AutoPLC: Generating Vendor-Aware Structured Text for Programmable Logic Controllers](https://mlsanity.com/paper/2412.02410/) (2024)
- [SemaPLC: A Project-Grounded, Verification-Gated Agent Harness for PLC Code Generation](https://mlsanity.com/paper/2608.18565/) (2026)

## Abstract (arXiv)

Programmable logic controllers (PLCs) coordinate industrial equipment by reading sensor inputs and issuing control commands. Evaluating whether large language model (LLM)-generated PLC programs satisfy task requirements and safety constraints requires observing how their commands affect device and workpiece states. We introduce PLCWorld, a common closed-loop execution environment and benchmark that couples Structured Text (ST) execution with simulated plant responses and sensor feedback. Grounded in control relations identified in industrial PLC programs and engineering documentation, PLCWorld contains 100 synthetic tasks and 473 registered task-condition pairs across Motion Control and Material Handling, with difficulty defined by control-dependency scope. A common protocol reports Task Success and Safety Violation separately. Validation combines practitioner review, reference and alternative programs, targeted counterexamples, specification-evaluator alignment checks, and comparisons with independent ST runtimes. Reference and alternative programs satisfy their applicable cases, while all 542 targeted counterexamples activate their designated evaluator rules under at least one registered condition. Execution Gap relates submission-profile acceptance to subsequent task failure or observed Safety Violation. Across the constructed task groups, direct GPT-5.5 achieves 82.70% Task Success on Easy cases but 25.10% on Hard cases. Evaluations of six LLMs and four adapted generation-and-verification workflows further expose differences between completion, safety, and generation cost. Our code, simulation environment, benchmark tasks, and baseline implementations are publicly available at https://yunji0516.github.io/PLCWorld/.