# Learning-based Bounded Synthesis for Semi-MDPs with LTL Specifications

arXiv: https://arxiv.org/abs/2204.04383 · HTML: https://mlsanity.com/paper/2204.04383/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2022.

## Built on by (most cited first, 1 of 1)

- [Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDP](https://mlsanity.com/paper/2303.09528/) (2023)

## Abstract (arXiv)

This letter proposes a learning-based bounded synthesis for a semi-Markov decision process (SMDP) with a linear temporal logic (LTL) specification. In the product of the SMDP and the deterministic K-co-B"uchi automaton (dKcBA) converted from the LTL specification, we learn both the winning region of satisfying the LTL specification and the dynamics therein based on reinforcement learning and Bayesian inference. Then, we synthesize an optimal policy satisfying the following two conditions. (1) It maximizes the probability of reaching the wining region. (2) It minimizes a long-term risk for the dwell time within the winning region. The minimization of the long-term risk is done based on the estimated dynamics and a value iteration. We show that, if the discount factor is sufficiently close to one, the synthesized policy converges to the optimal policy as the number of the data obtained by the exploration goes to the infinity.