mlsanityData to 7 Oct 2026

Multi-Task Reinforcement Learning with Context-based Representations

arXiv 2102.06177 ↗First posted 11 Feb 2021cs.LG cs.AI cs.ROcited by 169

Not on the map. Why?

Abstract (arXiv)

The benefit of multi-task learning over single-task learning relies on the ability to use relations across tasks to improve performance on any single task. While sharing representations is an important mechanism to share information across tasks, its success depends on how well the structure underlying the tasks is captured. In some real-world situations, we have access to metadata, or additional information about a task, that may not provide any new insight in the context of a single task setup alone but inform relations across multiple tasks. While this metadata can be useful for improving multi-task learning performance, effectively incorporating it can be an additional challenge. We posit that an efficient approach to knowledge transfer is through the use of multiple context-dependent, composable representations shared across a family of tasks. In this framework, metadata can help to learn interpretable representations and provide the context to inform which representations to compose and how to compose them. We use the proposed approach to obtain state-of-the-art results in Meta-World, a challenging multi-task benchmark consisting of 50 distinct robotic manipulation tasks.

Key facts. Cited by 169 papers in the map (data through 7 Oct 2026). Builds on 52 of its 84 references that are in the map. First posted on arXiv 11 Feb 2021.

Builds on

52 of 84 references.

References · 52

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2016OpenAI Gymcited by 1,684
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  9. 2018DeepMind Control Suitecited by 935
  10. 2020Object-Centric Learning with Slot Attentioncited by 865
  11. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  12. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
38 more
  1. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  2. 2017Deep Reinforcement Learning that Matterscited by 727
  3. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  4. 2019Model-Based Reinforcement Learning for Ataricited by 495
  5. 2015Policy Distillationcited by 414
  6. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  7. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  8. 2018Neural Relational Inference for Interacting Systemscited by 339
  9. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  10. 2019Recurrent Independent Mechanismscited by 219
  11. 2015Contextual Markov Decision Processescited by 213
  12. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  13. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  14. 2015The Benefit of Multitask Representation Learningcited by 194
  15. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  16. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  17. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  18. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  19. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  20. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  21. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  22. 2017Learning Independent Causal Mechanismscited by 104
  23. 2020Invariant Causal Prediction for Block MDPscited by 104
  24. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  25. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  26. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  27. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  28. 2020Causal Discovery in Physical Systems from Videoscited by 77
  29. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  30. 2019Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularitycited by 68
  31. 2019Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoningcited by 60
  32. 2017Markov Decision Processes with Continuous Side Informationcited by 54
  33. 2019Regularizing Deep Multi-Task Networks using Orthogonal Gradientscited by 36
  34. 2018Automatically Composing Representation Transformations as a Means for Generalizationcited by 35
  35. 2019Learning Causal State Representations of Partially Observable Environmentscited by 30
  36. 2019Self-Paced Contextual Reinforcement Learningcited by 26
  37. 2016Learning Shared Representations in Multi-task Reinforcement Learningcited by 15
  38. 2019Multitask Soft Option Learningcited by 13

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 30; 2023: 39; 2024: 38; 2025: 35; 2026: 22.

169 papers.

Sort:

By area:

  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  2. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  3. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  4. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  5. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  6. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  7. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  8. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 74
  9. 2023Independent Component Alignment for Multi-Task Learningcited by 69
  10. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 65
  11. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  12. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
38 more
  1. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  2. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
  3. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  4. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  5. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  6. 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 37
  7. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  8. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  9. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  10. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 28
  11. 2023Learning to Modulate pre-trained Models in RLcited by 28
  12. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  13. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  14. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  15. 2021Block Contextual MDPs for Continual Learningcited by 25
  16. 2023Provable Reward-Agnostic Preference-Based Reinforcement Learningcited by 25
  17. 2021Metadata-based Multi-Task Bandits with Bayesian Hierarchical Modelscited by 24
  18. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  19. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  20. 2023Universal Morphology Control via Contextual Modulationcited by 21
  21. 2024Natural Language Can Help Bridge the Sim2Real Gapcited by 21
  22. 2022Language Control Diffusion: Efficiently Scaling through Space, Time, and Taskscited by 20
  23. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  24. 2022Improving Policy Optimization with Generalist-Specialist Learningcited by 19
  25. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  26. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
  27. 2022Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Taskscited by 17
  28. 2024Semantically Controllable Augmentations for Generalizable Robot Learningcited by 17
  29. 2025MoE-Loco: Mixture of Experts for Multitask Locomotioncited by 17
  30. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  31. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  32. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  33. 2022Learning Mixtures of Linear Dynamical Systemscited by 15
  34. 2023Provably Feedback-Efficient Reinforcement Learning via Active Reward Learningcited by 15
  35. 2022Multi-Agent Policy Transfer via Task Relationship Modelingcited by 14
  36. 2024Offline Multitask Representation Learning for Reinforcement Learningcited by 14
  37. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  38. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 14

50 most cited of 169 All 169 (JSON)