# Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processes

arXiv: https://arxiv.org/abs/1706.06544 · HTML: https://mlsanity.com/paper/1706.06544/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2017.

## Built on by (most cited first, 29 of 29)

- [Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.05265/) (2021)
- [Federated Reinforcement Learning with Environment Heterogeneity](https://mlsanity.com/paper/2204.02634/) (2022)
- [Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations](https://mlsanity.com/paper/2206.04779/) (2022)
- [Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment](https://mlsanity.com/paper/2104.05632/) (2021)
- [Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models](https://mlsanity.com/paper/2102.08291/) (2021)
- [Model-based Lifelong Reinforcement Learning with Bayesian Exploration](https://mlsanity.com/paper/2210.11579/) (2022)
- [medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support](https://mlsanity.com/paper/2505.19785/) (2025)
- [Model-based Reinforcement Learning for Parameterized Action Spaces](https://mlsanity.com/paper/2404.03037/) (2024)
- [Meta-Learning Parameterized Skills](https://mlsanity.com/paper/2206.03597/) (2022)
- [Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayes Theory](https://mlsanity.com/paper/2411.00401/) (2024)
- [Act as You Learn: Adaptive Decision-Making in Non-Stationary Markov Decision Processes](https://mlsanity.com/paper/2401.01841/) (2024)
- [DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning](https://mlsanity.com/paper/2402.15957/) (2024)
- [Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles with Joint Radar-Data Communications](https://mlsanity.com/paper/2105.13670/) (2021)
- [Introspective Action Advising for Interpretable Transfer Learning](https://mlsanity.com/paper/2306.12314/) (2023)
- [R3L: Relative Representations for Reinforcement Learning](https://mlsanity.com/paper/2404.12917/) (2024)
- [Multi-task Batch Reinforcement Learning with Metric Learning](https://mlsanity.com/paper/1909.11373/) (2019)
- [Meta-Learning for Koopman Spectral Analysis with Short Time-series](https://mlsanity.com/paper/2102.04683/) (2021)
- [Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning](https://mlsanity.com/paper/2112.15221/) (2021)
- [Learning and Understanding a Disentangled Feature Representation for Hidden Parameters in Reinforcement Learning](https://mlsanity.com/paper/2211.16315/) (2022)
- [Policy Resilience to Environment Poisoning Attacks on Reinforcement Learning](https://mlsanity.com/paper/2304.12151/) (2023)
- [Situationally-Aware Dynamics Learning](https://mlsanity.com/paper/2505.19574/) (2025)
- [Compositional Q-learning for electrolyte repletion with imbalanced patient sub-populations](https://mlsanity.com/paper/2110.02879/) (2021)
- [On the Benefits of Leveraging Structural Information in Planning Over the Learned Model](https://mlsanity.com/paper/2303.08856/) (2023)
- [Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation](https://mlsanity.com/paper/2310.14976/) (2023)
- [Mapping representations in Reinforcement Learning via Semantic Alignment for Zero-Shot Stitching](https://mlsanity.com/paper/2503.01881/) (2025)
- [Meta-RL with Bayesian Linear Task Models](https://mlsanity.com/paper/2512.20974/) (2025)
- [When Does Test-Time Physical Diagnosis Pay? A Frozen Policy Buys Evidence It Never Reads](https://mlsanity.com/paper/2609.22299/) (2026)
- [Robust Successor Features](https://mlsanity.com/paper/2609.31016/) (2026)
- [Learning Transferable Policies from Action-free Time Series Through Dynamical Embeddings](https://mlsanity.com/paper/2610.03065/) (2026)

## Abstract (arXiv)

We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty in the latent parameters and the state space. We also replace the original Gaussian Process-based model with a Bayesian Neural Network, enabling more scalable inference. Thus, we expand the scope of the HiP-MDP to applications with higher dimensions and more complex dynamics.