# PAC Bounds for Imitation and Model-based Batch Learning of Contextual Markov Decision Processes

arXiv: https://arxiv.org/abs/2006.06352 · HTML: https://mlsanity.com/paper/2006.06352/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 1 of 1)

- [Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)](https://mlsanity.com/paper/2109.14678/) (2021)

## Abstract (arXiv)

We consider the problem of batch multi-task reinforcement learning with observed context descriptors, motivated by its application to personalized medical treatment. In particular, we study two general classes of learning algorithms: direct policy learning (DPL), an imitation-learning based approach which learns from expert trajectories, and model-based learning. First, we derive sample complexity bounds for DPL, and then show that model-based learning from expert actions can, even with a finite model class, be impossible. After relaxing the conditions under which the model-based approach is expected to learn by allowing for greater coverage of state-action space, we provide sample complexity bounds for model-based learning with finite model classes, showing that there exist model classes with sample complexity exponential in their statistical complexity. We then derive a sample complexity upper bound for model-based learning based on a measure of concentration of the data distribution. Our results give formal justification for imitation learning over model-based learning in this setting.