mlsanityData to 14 Oct 2026

Temporal Property-driven Design Space Exploration with Reinforcement Learning for Cyber-Physical Systems

arXiv 2608.23440 ↗First posted 24 Aug 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Design-space exploration of configurable Cyber-Physical Systems (CPS) requires executable evaluation when design choices affect timing, fault propagation, recovery behavior, and temporal-property satisfaction. Repeated stochastic executions make exhaustive exploration impractical for large design spaces. This paper presents a temporal-property-driven CPS design workflow using Reinforcement Learning (RL). At design time, the RL agent selects subsystem alternatives to assemble a candidate system model. The model is then evaluated through simulation, during which online temporal-property monitors observe runtime traces and produce functional-property violation indicators. These indicators are combined with evaluated non-functional terms for budget, recoverability, sustained compliance, and operational use to calculate the reward used for subsequent candidate selection. The workflow is evaluated on a methane-sensitive mine-pump CPS. The corresponding executable case-study model is provided as additional contribution. RL-guided search identifies the highest-reward design observed in the experiments after 26 episodes (corresponds to 130 executable simulations). These designs were reached with fewer simulations than surrogate-guided Bayesian Optimization and population-based Genetic Algorithm baselines under the same executable model and reward formulation. Ablation study results indicate that value-based feedback and reuse of previous simulation traces contribute to this reduction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 2 of its 21 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

2 of 21 references.

References · 2

  1. 2021Model-based Fault Injection Experiments for the Safety Analysis of Exoskeleton Systemcited by 1
  2. 2024Optimizing Feature Selection with Genetic Algorithms: A Review of Methods and Applicationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Control Synthesis of Cyber-Physical Systems for Real-Time Specifications through Causation-Guided Reinforcement Learning
  2. 2025Evaluating Reinforcement Learning Safety and Trustworthiness in Cyber-Physical Systems
  3. 2025Logic-informed reinforcement learning for cross-domain optimization of large-scale cyber-physical systems
  4. 2026Strategic Evaluation of Planning Strategies for LLM Agents in Cyber-Physical Systems
  5. 2025The Crucial Role of Problem Formulation in Real-World Reinforcement Learning
  6. 2025Surrogate Fitness Metrics for Interpretable Reinforcement Learning
  7. 2025Input-Output Data-Driven Sensor Selection for Cyber-Physical Systems ↗
  8. 2026SMC-ES: Automated synthesis of formally verified control policies