mlsanityData to 14 Oct 2026

Adaptive Policy Backbone via Shared Network

arXiv 2509.22310 ↗First posted 26 Sep 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning (RL) has achieved impressive results across domains, yet learning an optimal policy typically requires extensive interaction data, limiting practical deployment. A common remedy is to leverage priors, such as pre-collected datasets or reference policies, but their utility degrades under task mismatch between training and deployment. While prior work has sought to address this mismatch, it has largely been restricted to in-distribution settings. To address this challenge, we propose Adaptive Policy Backbone (APB), a meta-transfer RL method that inserts lightweight linear layers before and after a shared backbone, thereby enabling parameter-efficient fine-tuning (PEFT) while preserving prior knowledge during adaptation. Our results show that APB improves sample efficiency over standard RL and adapts to out-of-distribution (OOD) tasks where existing meta-RL baselines typically fail.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 56 of its 74 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

56 of 74 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning25 Feb
  2. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learning17 Jan

Other references · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2020Learning to summarize from human feedbackcited by 2,588
  9. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  12. 2022Visual Prompt Tuningcited by 1,681
38 more
  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  2. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  8. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  10. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  12. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  13. 2019Solving Rubik's Cube with a Robot Handcited by 734
  14. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  15. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  16. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  17. 2016Learning to reinforcement learncited by 377
  18. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  19. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
  20. 2017Meta Networkscited by 269
  21. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  22. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  23. 2017Parameter Space Noise for Explorationcited by 161
  24. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 158
  25. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  26. 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 141
  27. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  28. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 127
  29. 2018Universal Successor Features Approximatorscited by 103
  30. 2019ES-MAML: Simple Hessian-Free Meta Learningcited by 57
  31. 2018Some Considerations on Learning to Explore via Meta-Reinforcement Learningcited by 44
  32. 2020Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agentscited by 35
  33. 2023Lossless Adaptation of Pretrained Vision Models For Robotic Manipulationcited by 29
  34. 2017Learning to Learn: Meta-Critic Networks for Sample Efficient Learningcited by 28
  35. 2023Learning to Modulate pre-trained Models in RLcited by 28
  36. 2019Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspectivecited by 27
  37. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  38. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 17

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training
  2. 2024Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator
  3. 2026APC-RL: Exceeding Data-Driven Behavior Priors with Adaptive Policy Composition
  4. 2025Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimation
  5. 2026Lever: Inference-Time Policy Reuse under Support Constraints
  6. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone
  7. 2024Theoretical Analysis of Meta Reinforcement Learning: Generalization Bounds and Convergence Guarantees
  8. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correction