mlsanityData to 7 Oct 2026

Imitation Learning from Pixel-Level Demonstrations by HashReward

arXiv 1909.03773 ↗First posted 9 Sep 2019cs.LG stat.MLcited by 6

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

One of the key issues for imitation learning lies in making policy learned from limited samples to generalize well in the whole state-action space. This problem is much more severe in high-dimensional state environments, such as game playing with raw pixel inputs. Under this situation, even state-of-the-art adversary-based imitation learning algorithms fail. Through empirical studies, we find that the main cause lies in the failure of training a powerful discriminator to generate meaningful rewards in high-dimensional environments. Although it seems that dimensionality reduction can help, a straightforward application of off-the-shelf methods cannot achieve good performance. In this work, we show in theory that the balance between dimensionality reduction and discriminative training is essential for effective learning. To achieve this target, we propose HashReward, which utilizes the idea of supervised hashing to realize such an ideal balance. Experimental results show that HashReward could outperform state-of-the-art methods for a large gap under the challenging high-dimensional environments.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 30 references that are in the map. First posted on arXiv 9 Sep 2019.

Builds on

19 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct

Other references · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2016OpenAI Gymcited by 1,684
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  8. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  9. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  10. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  11. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  12. 2016Model-Free Episodic Controlcited by 100
6 more
  1. 2018Playing hard exploration games by watching YouTubecited by 98
  2. 2018Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learningcited by 80
  3. 2017On the Discrimination-Generalization Tradeoff in GANscited by 48
  4. 2017Asymmetric Deep Supervised Hashingcited by 35
  5. 2018Inverse reinforcement learning for video gamescited by 26
  6. 2017Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 2.

6 papers.

  1. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  2. 2021Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learningcited by 4
  3. 2024Enhancing Reinforcement Learning Agents with Local Guidescited by 4
  4. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learningcited by 3
  5. 2023Imitation Learning from Purified Demonstrationscited by 3
  6. 2024Visual Imitation Learning with Calibrated Contrastive Representationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Sample-efficient Adversarial Imitation Learning
  2. 2021Provable Representation Learning for Imitation with Contrastive Fourier Features
  3. 2024Efficient Imitation Learning with Conservative World Models
  4. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  5. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
  6. 2022A Ranking Game for Imitation Learning
  7. 2026Difference-Aware Retrieval Policies for Imitation Learning
  8. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances