mlsanityData to 14 Oct 2026

Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routing

arXiv 2312.14472 ↗First posted 22 Dec 2023cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Multi-task reinforcement learning endeavors to accomplish a set of different tasks with a single policy. To enhance data efficiency by sharing parameters across multiple tasks, a common practice segments the network into distinct modules and trains a routing network to recombine these modules into task-specific policies. However, existing routing approaches employ a fixed number of modules for all tasks, neglecting that tasks with varying difficulties commonly require varying amounts of knowledge. This work presents a Dynamic Depth Routing (D2R) framework, which learns strategic skipping of certain intermediate modules, thereby flexibly choosing different numbers of modules for each task. Under this framework, we further introduce a ResRouting method to address the issue of disparate routing paths between behavior and target policies during off-policy training. In addition, we design an automatic route-balancing mechanism to encourage continued routing exploration for unmastered tasks without disturbing the routing of mastered ones. We conduct extensive experiments on various robotics manipulation tasks in the Meta-World benchmark, where D2R achieves state-of-the-art performance with significantly improved learning efficiency.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 22 Dec 2023.

Builds on

23 of 34 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  12. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 364
11 more
  1. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
  2. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  3. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 178
  4. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  5. 2017Adaptive Neural Networks for Efficient Inferencecited by 163
  6. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  7. 2017Convolutional Networks with Adaptive Inference Graphscited by 148
  8. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  9. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  10. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  11. 2020Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Controlcited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  2. 2025Efficient Multi-Task Reinforcement Learning with Cross-Task Policy Guidancecited by 3
  3. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planningcited by 3
  4. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learningcited by 2
  5. 2024Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learningcited by 1
  6. 2024Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 1
  7. 2026Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamicscited by 1
  8. 2025Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learningcited by 0