Reinforced Imitation in Heterogeneous Action Space
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse. In this paper, we consider a challenging setting where an agent and an expert use different actions from each other. We assume that the agent has access to a sparse reward function and state-only expert observations. We propose a method which gradually balances between the imitation learning cost and the reinforcement learning objective. In addition, this method adapts the agent's policy based on either mimicking expert behavior or maximizing sparse reward. We show, through navigation scenarios, that (i) an agent is able to efficiently leverage sparse rewards to outperform standard state-only imitation learning, (ii) it can learn a policy even when its actions are different from the expert, and (iii) the performance of the agent is not bounded by that of the expert, due to the optimized usage of sparse rewards.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 6 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2019: 2; 2020: 1; 2021: 0; 2022: 4; 2023: 0; 2024: 1.
8 papers.
- 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
- 2019Task-Relevant Adversarial Imitation Learningcited by 45
- 2019Positive-Unlabeled Reward Learningcited by 36
- 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
- 2022Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?cited by 4
- 2022Self-Imitation Learning from Demonstrationscited by 4
- 2022D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioningcited by 2
- 2024Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mappingcited by 0
- 2024Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mapping18 Oct 2024
- 2022D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioning26 Oct 2022
- 2022Imitation Learning from Observations under Transition Model Disparity25 Apr 2022
- 2022Self-Imitation Learning from Demonstrations21 Mar 2022
- 2022Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?24 Feb 2022
- 2020Offline Learning from Demonstrations and Unlabeled Experience27 Nov 2020
- 2019Positive-Unlabeled Reward Learning1 Nov 2019
- 2019Task-Relevant Adversarial Imitation Learning2 Oct 2019
Similar, not cited
Close in wording, no citation either way
- 2021Imitation Learning by Reinforcement Learning
- 2022Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation
- 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
- 2024Align Your Intents: Offline Imitation Learning via Optimal Transport
- 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
- 2022A Ranking Game for Imitation Learning
- 2021Hyperparameter Selection for Imitation Learning
- 2022Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning