# Self-Triggered Markov Decision Processes

arXiv: https://arxiv.org/abs/2102.08571 · HTML: https://mlsanity.com/paper/2102.08571/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 25 references that are in the map.
- First posted on arXiv 17 Feb 2021.

## Builds on (most cited first, 5 of 5)

- [Continuous-Time Markov Decision Processes with Controlled Observations](https://mlsanity.com/paper/1907.06128/) (2019)
- [A Pursuit-Evasion Differential Game with Strategic Information Acquisition](https://mlsanity.com/paper/2102.05469/) (2021)
- [Optimality Conditions for Inventory Control](https://mlsanity.com/paper/1606.00957/) (2016)
- [Cross-Layer Coordinated Attacks on Cyber-Physical Systems: A LQG Game Framework with Controlled Observations](https://mlsanity.com/paper/2012.02384/) (2020)
- [Infinite-Horizon Linear-Quadratic-Gaussian Control with Costly Measurements](https://mlsanity.com/paper/2012.14925/) (2020)

## Abstract (arXiv)

In this paper, we study Markov Decision Processes (MDPs) with self-triggered strategies, where the idea of self-triggered control is extended to more generic MDP models. This extension broadens the application of self-triggering policies to a broader range of systems. We study the co-design problems of the control policy and the triggering policy to optimize two pre-specified cost criteria. The first cost criterion is introduced by incorporating a pre-specified update penalty into the traditional MDP cost criteria to reduce the use of communication resources. Under this criteria, a novel dynamic programming (DP) equation called DP equation with optimized lookahead to proposed to solve for the self-triggering policy under this criteria. The second self-triggering policy is to maximize the triggering time while still guaranteeing a pre-specified level of sub-optimality. Theoretical underpinnings are established for the computation and implementation of both policies. Through a gridworld numerical example, we illustrate the two policies' effectiveness in reducing sources consumption and demonstrate the trade-offs between resource consumption and system performance.