# Comparing discounted and average-cost Markov Decision Processes: a statistical significance perspective

arXiv: https://arxiv.org/abs/2112.00684 · HTML: https://mlsanity.com/paper/2112.00684/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2021.

## Built on by (most cited first, 1 of 1)

- [Solving the non-preemptive two queue polling model with generally distributed service and switch-over durations and Poisson arrivals as a Semi-Markov Decision Process](https://mlsanity.com/paper/2112.06578/) (2021)

## Abstract (arXiv)

Optimal Markov Decision Process policies for problems with finite state and action space are identified through a partial ordering by comparing the value function across states. This is referred to as state-based optimality. This paper identifies when such optimality guarantees some form of system-based optimality as measured by a scalar. Four such system-based metrics are introduced. Uni-variate empirical distributions of these metrics are obtained through simulation as to assess whether theoretically optimal policies provide a statistically significant advantage. This has been conducted using a Student's t-test, Welch's t-test and a Mann-Whitney U-test. The proposed method is applied to a common problem in queuing theory: admission control.