mlsanityData to 7 Oct 2026

On the Power of Pre-training for Generalization in RL: Provable Benefits and Hardness

arXiv 2210.10464 ↗First posted 19 Oct 2022cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Generalization in Reinforcement Learning (RL) aims to learn an agent during training that generalizes to the target environment. This paper studies RL generalization from a theoretical aspect: how much can we expect pre-training over training environments to be helpful? When the interaction with the target environment is not allowed, we certify that the best we can obtain is a near-optimal policy in an average sense, and we design an algorithm that achieves this goal. Furthermore, when the agent is allowed to interact with the target environment, we give a surprising result showing that asymptotically, the improvement from pre-training is at most a constant factor. On the other hand, in the non-asymptotic regime, we design an efficient algorithm and prove a distribution-based regret bound in the target environment that is independent of the state-action space.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 40 of its 64 references that are in the map. First posted on arXiv 19 Oct 2022.

Builds on

40 of 64 references.

References · 40

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2018Learning Dexterous In-Hand Manipulationcited by 643
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  4. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  5. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  6. 2018Is Q-learning Provably Efficient?cited by 485
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  8. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  11. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  12. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
28 more
  1. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  2. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  3. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  4. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 171
  5. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  6. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  7. 2017Generalization in Deep Learningcited by 146
  8. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  9. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  10. 2018Generalization and Regularization in DQNcited by 117
  11. 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 116
  12. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
  13. 2020Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizoncited by 99
  14. 2017Real-Time Bidding by Reinforcement Learning in Display Advertisingcited by 90
  15. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
  16. 2020Sample-Efficient Reinforcement Learning of Undercomplete POMDPscited by 66
  17. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  18. 2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
  19. 2021Risk Bounds and Rademacher Complexity in Batch Reinforcement Learningcited by 42
  20. 2021Nearly Horizon-Free Offline Reinforcement Learningcited by 36
  21. 2018Variational Bayesian Reinforcement Learning with Regret Boundscited by 31
  22. 2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
  23. 2019On the Generalization Gap in Reparameterizable Reinforcement Learningcited by 21
  24. 2021When Is Generalizable Reinforcement Learning Tractable?cited by 19
  25. 2021Settling the Horizon-Dependence of Sample Complexity in Reinforcement Learningcited by 17
  26. 2021Provably Efficient Multi-Task Reinforcement Learning with Model Transfercited by 16
  27. 2021Near-Optimal Reward-Free Exploration for Linear Mixture MDPs with Plug-in Solvercited by 13
  28. 2020Sequential Transfer in Reinforcement Learning with a Generative Modelcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RLcited by 17
  2. 2024Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Datacited by 8
  3. 2024Test-Time Regret Minimization in Meta Reinforcement Learningcited by 5
  4. 2025Provable Zero-Shot Generalization in Offline Reinforcement Learningcited by 3
  5. 2025A Classification View on Meta Learning Banditscited by 1
  6. 2026Constrained Meta Reinforcement Learning with Provable Test-Time Safetycited by 1
  7. 2026Provably Safe Sim-to-Real Transfercited by 0