# MDPs with Setwise Continuous Transition Probabilities

arXiv: https://arxiv.org/abs/2011.01325 · HTML: https://mlsanity.com/paper/2011.01325/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity](https://mlsanity.com/paper/2111.06781/) (2021)
- [Compactly Restrictable Metric Policy Optimization Problems](https://mlsanity.com/paper/2207.05850/) (2022)
- [Threshold Structure of Optimal Policies in Restart POMDPs](https://mlsanity.com/paper/2608.10936/) (2026)

## Abstract (arXiv)

This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.