Task-Agnostic Learning to Accomplish New Tasks

arXiv 2209.04100 ↗First posted 9 Sep 2022cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Reinforcement Learning (RL) and Imitation Learning (IL) have made great progress in robotic decision-making in recent years. However, these methods show obvious deterioration for new tasks that need to be completed through new combinations of actions. RL methods suffer from reward functions and distribution shifts, while IL methods are limited by expert demonstrations which do not cover new tasks. In contrast, humans can easily complete these tasks with the fragmented knowledge learned from task-agnostic experience. Inspired by this observation, this paper proposes a task-agnostic learning method (TAL for short) that can learn fragmented knowledge only from task-agnostic data to accomplish new tasks. TAL consists of four stages. First, the task-agnostic exploration is performed to collect data from interactions with the environment. The collected data is organized via a knowledge graph. Second, an action feature extractor is proposed and trained using the collected knowledge graph data for task-agnostic fragmented knowledge learning. Third, a candidate action generator is designed, which applies the action feature extractor on a new task to generate multiple candidate action sets. Finally, an action proposal network is designed to produce the probabilities for actions in a new task according to the environmental information. The probabilities are then used to generate order information for selecting actions to be executed from multiple candidate action sets to form the plan. Experiments on a virtual indoor scene show that the proposed method outperforms the state-of-the-art offline RL methods and IL methods by more than 20%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 55 of its 100 references that are in the map. First posted on arXiv 9 Sep 2022.

Builds on

55 of 100 references.

Closest ancestors · 2

Same direction, newest first

Other references · 53

2017Proximal Policy Optimization Algorithmscited by 14,441
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
2015Continuous control with deep reinforcement learningcited by 3,279
2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
2016Generative Adversarial Imitation Learningcited by 1,576
2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
2018Exploration by Random Network Distillationcited by 966
2020Meta-Learning in Neural Networks: A Surveycited by 944

38 more

2018Behavioral Cloning from Observationcited by 482
2018An Algorithmic Perspective on Imitation Learningcited by 455
2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
2020Planning to Explore via Self-Supervised World Modelscited by 365
2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
2019Learning Latent Plans from Playcited by 310
2016Deep Visual Foresight for Planning Robot Motioncited by 276
2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
2017Inverse Reward Designcited by 246
2020Never Give Up: Learning Directed Exploration Strategiescited by 210
2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
2020Reward-Free Exploration for Reinforcement Learningcited by 177
2021A Fine-Grained Analysis on Distribution Shiftcited by 171
2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
2021Discovering and Achieving Goals via World Modelscited by 131
2019Imitation Learning from Imperfect Demonstrationcited by 112
2018Imitating Latent Policies from Observationcited by 101
2019Recent Advances in Imitation Learning from Observationcited by 92
2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
2020Learning to Continually Learncited by 82
2017Learning human behaviors from motion capture by adversarial imitationcited by 75
2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
2021Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learningcited by 42
2020Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effectscited by 40
2018Deep Reinforcement Learning For Sequence to Sequence Modelscited by 39
2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
2020Task-agnostic Exploration in Reinforcement Learningcited by 37
2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
2020Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environmentcited by 29
2021Learning from Imperfect Demonstrations from Agents with Varying Dynamicscited by 26
2019Variational Metric Scaling for Metric-Based Meta-Learningcited by 17
2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 10

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way