mlsanityData to 14 Oct 2026

Meta Learning MDPs with Linear Transition Models

arXiv 2201.08732 ↗First posted 21 Jan 2022cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We study meta-learning in Markov Decision Processes (MDP) with linear transition models in the undiscounted episodic setting. Under a task sharedness metric based on model proximity we study task families characterized by a distribution over models specified by a bias term and a variance component. We then propose BUC-MatrixRL, a version of the UC-Matrix RL algorithm, and show it can meaningfully leverage a set of sampled training tasks to quickly solve a test task sampled from the same task distribution by learning an estimator of the bias parameter of the task distribution. The analysis leverages and extends results in the learning to learn linear regression and linear bandit setting to the more general case of MDP's with linear transition models. We prove that compared to learning the tasks in isolation, BUC-Matrix RL provides significant improvements in the transfer regret for high bias low variance task distributions.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 21 Jan 2022.

Builds on

23 of 39 references.

References · 23

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  3. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  4. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  5. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  6. 2019Provably Efficient Exploration in Policy Optimizationcited by 205
  7. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  8. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 200
  9. 2020Provable Meta-Learning of Linear Representationscited by 161
  10. 2021Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraintscited by 124
  11. 2020Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mappingcited by 102
  12. 2019Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensemblescited by 100
11 more
  1. 2020Logarithmic Regret for Reinforcement Learning with Linear Function Approximationcited by 79
  2. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  3. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  4. 2020Impact of Representation Learning in Linear Banditscited by 52
  5. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  6. 2017Multi-Task Learning for Contextual Banditscited by 47
  7. 2020Meta-learning with Stochastic Linear Banditscited by 46
  8. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  9. 2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
  10. 2021When Is Generalizable Reinforcement Learning Tractable?cited by 20
  11. 2020A Distribution-Dependent Analysis of Meta-Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 0; 2024: 1.

4 papers.

  1. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  2. 2024Offline Multitask Representation Learning for Reinforcement Learningcited by 14
  3. 2022Joint Representation Training in Sequential Tasks with Shared Structurecited by 5
  4. 2022POMRL: No-Regret Learning-to-Plan with Increasing Horizonscited by 0