# Transfer Learning Across Patient Variations with Hidden Parameter Markov Decision Processes

arXiv: https://arxiv.org/abs/1612.00475 · HTML: https://mlsanity.com/paper/1612.00475/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2016.

## Built on by (most cited first, 2 of 2)

- [Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios](https://mlsanity.com/paper/2206.14697/) (2022)
- [Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective](https://mlsanity.com/paper/2404.16078/) (2024)

## Abstract (arXiv)

Due to physiological variation, patients diagnosed with the same condition may exhibit divergent, but related, responses to the same treatments. Hidden Parameter Markov Decision Processes (HiP-MDPs) tackle this transfer-learning problem by embedding these tasks into a low-dimensional space. However, the original formulation of HiP-MDP had a critical flaw: the embedding uncertainty was modeled independently of the agent's state uncertainty, requiring an unnatural training procedure in which all tasks visited every part of the state space---possible for robots that can be moved to a particular location, impossible for human patients. We update the HiP-MDP framework and extend it to more robustly develop personalized medicine strategies for HIV treatment.