On Completely Mixed Stochastic Games

In this paper, we consider a zero-sum undiscounted stochastic game which has finite state space and finitely many pure actions. Also, we assume the transition probability of the undiscounted stochastic game is controlled by one player and all the optimal strategies of the game are strictly positive. Under all the above assumptions, we show that the β-discounted stochastic games with the same payoff matrices and β sufficiently close to 1 are also completely mixed. We give a counterexample to show that the converse of the above result in not true. We also show that, if we have non-zero value in some state for the undiscounted stochastic game then for β sufficiently close to 1 the β-discounted stochastic game also possess nonzero value in the same state.