# Grammar-Constrained Refinement of Safety Operational Rules Using Language in the Loop: What Could Go Wrong

arXiv: https://arxiv.org/abs/2604.23523 · HTML: https://mlsanity.com/paper/2604.23523/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 25 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Builds on (most cited first, 6 of 6)

- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1804.06760/) (2018)
- [Mining Parametric Temporal Logic Properties in Model Based Design for Cyber-Physical Systems](https://mlsanity.com/paper/1512.07956/) (2015)
- [Test Generation Strategies for Building Failure Models and Explaining Spurious Failures](https://mlsanity.com/paper/2312.05631/) (2023)
- [Combining Genetic Programming and Model Checking to Generate Environment Assumptions](https://mlsanity.com/paper/2101.01933/) (2021)
- [Automatic Generation of Safety-compliant Linear Temporal Logic via Large Language Model: A Self-supervised Framework](https://mlsanity.com/paper/2503.15840/) (2025)

## Built on by (most cited first, 1 of 1)

- [Teach-to-Crash: A Closed-Loop Student-Teacher LLM Framework for Collision-Inducing Test Scenario Generation](https://mlsanity.com/paper/2609.27296/) (2026)

## Abstract (arXiv)

Safety specifications in cyber-physical systems (CPS) capture the operational conditions the system must satisfy to operate safely within its intended environment. As operating environments evolve, operational rules must be continuously refined to preserve consistency with observed system behavior during simulation-based verification and validation. Revising inconsistent rules is challenging because the changes must remain syntactically correct under a domain-specific grammar. Language-in-the-loop refinement further raises safety concerns beyond syntactic violations, as it can produce semantically unjustified refinements that overfit to the observed outcomes. We introduce a framework that combines counterfactual reasoning with a grammar-constrained refinement loop to refine operational rules, aligning them with the observed system behavior. Applied to an autonomous driving control system, our approach successfully resolved the inconsistencies in an operational rule inferred by a conventional baseline while remaining grammar compliant. An empirical large language model (LLM) study further revealed model-dependent refinement quality and safety lessons, which motivate rigorous grammar enforcement, stronger semantic validation, and broader evaluation in future work.