Learning Multi-Objective Curricula for Robotic Policy Learning
Not on the map. Why?
Abstract (arXiv)
Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL). They are designed to control how a DRL agent collects data, which is inspired by how humans gradually adapt their learning processes to their capabilities. For example, ACL can be used for subgoal generation, reward shaping, environment generation, or initial state generation. However, prior work only considers curriculum learning following one of the aforementioned predefined paradigms. It is unclear which of these paradigms are complementary, and how the combination of them can be learned from interactions with the environment. Therefore, in this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum. In order to coordinate those potentially conflicting modules in unified parameter space, we propose a multi-task hyper-net learning framework that uses a single hyper-net to parameterize all those curriculum modules. In addition to existing hand-designed curricula paradigms, we further design a flexible memory mechanism to learn an abstract curriculum, which may otherwise be difficult to design manually. We evaluate our method on a series of robotic manipulation tasks and demonstrate its superiority over other state-of-the-art ACL methods in terms of sample efficiency and final performance.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 6 Oct 2021.
Builds on
45 of 64 references.
References · 45
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2016Progressive Neural Networkscited by 1,332
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2016HyperNetworkscited by 1,003
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
- 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
- 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
33 more
- 2015Policy Distillationcited by 414
- 2021Dynamic Neural Networks: A Surveycited by 405
- 2016Dynamic Filter Networkscited by 318
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
- 2017Automated Curriculum Learning for Neural Networkscited by 251
- 2017Distral: Robust Multitask Reinforcement Learningcited by 231
- 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
- 2017Teacher-Student Curriculum Learningcited by 197
- 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
- 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
- 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
- 2017Neural Episodic Controlcited by 148
- 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
- 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
- 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
- 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
- 2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
- 2018Model-Based Active Explorationcited by 105
- 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 105
- 2016Model-Free Episodic Controlcited by 100
- 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
- 2019Generalized Inner Loop Meta-Learningcited by 92
- 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
- 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 83
- 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
- 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 70
- 2016Learning to Push by Grasping: Using multiple tasks for effective learningcited by 36
- 2018Learning Curriculum Policies for Reinforcement Learningcited by 34
- 2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 21
- 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20
- 2017Learning to Compose Skillscited by 15
- 2020Meta Automatic Curriculum Learningcited by 6
- 2019Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learningcited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 2.
3 papers.