# Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model

arXiv: https://arxiv.org/abs/2604.16111 · HTML: https://mlsanity.com/paper/2604.16111/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Builds on (most cited first, 10 of 10)

- [Minimax Regret Bounds for Reinforcement Learning](https://mlsanity.com/paper/1703.05449/) (2017)
- [Is Q-learning Provably Efficient?](https://mlsanity.com/paper/1807.03765/) (2018)
- [Model-Based Reinforcement Learning with a Generative Model is Minimax Optimal](https://mlsanity.com/paper/1906.03804/) (2019)
- [Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model](https://mlsanity.com/paper/2005.12900/) (2020)
- [Conservative Contextual Linear Bandits](https://mlsanity.com/paper/1611.06426/) (2016)
- [Variance Reduced Value Iteration and Faster Algorithms for Solving Markov Decision Processes](https://mlsanity.com/paper/1710.09988/) (2017)
- [Near-optimal Regret Bounds for Stochastic Shortest Path](https://mlsanity.com/paper/2002.09869/) (2020)
- [No-Regret Exploration in Goal-Oriented Reinforcement Learning](https://mlsanity.com/paper/1912.03517/) (2019)
- [Improved Analysis of UCRL2 with Empirical Bernstein Inequality](https://mlsanity.com/paper/2007.05456/) (2020)
- [Improved Sample Complexity for Incremental Autonomous Exploration in MDPs](https://mlsanity.com/paper/2012.14755/) (2020)

## Built on by (most cited first, 1 of 1)

- [Stochastic Reset Pathfinding: Path-Level Regret for Cascading Bandits over Graph Paths](https://mlsanity.com/paper/2607.15440/) (2026)

## Abstract (arXiv)

We study the sample complexity of learning an ε-optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bounds when the learner has access to a generative model. We show that there exists a worst-case SSP instance with S states, A actions, minimum cost cmin, and maximum expected cost of the optimal policy over all states Bstar, where any algorithm requires at least Omega(SABstar3/(cminε2)) samples to return an ε-optimal policy with high probability. Surprisingly, this implies that whenever cmin = 0 an SSP problem may not be learnable, thus revealing that learning in SSPs is strictly harder than in the finite-horizon and discounted settings. We complement this lower bound with an algorithm that matches it, up to logarithmic factors, in the general case, and an algorithm that matches it up to logarithmic factors even when cmin = 0, but only under the condition that the optimal policy has a bounded hitting time to the goal state.