mlsanityData to 7 Oct 2026

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation

arXiv 2203.03078 ↗First posted 7 Mar 2022cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

We present Nonparametric Approximation of Inter-Trace returns (NAIT), a Reinforcement Learning algorithm for discrete action, pixel-based environments that is both highly sample and computation efficient. NAIT is a lazy-learning approach with an update that is equivalent to episodic Monte-Carlo on episode completion, but that allows the stable incorporation of rewards while an episode is ongoing. We make use of a fixed domain-agnostic representation, simple distance based exploration and a proximity graph-based lookup to facilitate extremely fast execution. We empirically evaluate NAIT on both the 26 and 57 game variants of ATARI100k where, despite its simplicity, it achieves competitive performance in the online setting with greater than 100x speedup in wall-time.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 7 Mar 2022.

Builds on

22 of 38 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2017Deep Reinforcement Learning that Matterscited by 727
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  10. 2019Model-Based Reinforcement Learning for Ataricited by 495
  11. 2020Reinforcement Learning with Augmented Datacited by 452
  12. 2017Deep Reinforcement Learning: An Overviewcited by 327
10 more
  1. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  2. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  3. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  4. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  5. 2019When to use parametric models in reinforcement learning?cited by 118
  6. 2016Model-Free Episodic Controlcited by 100
  7. 2015Gated Feedback Recurrent Neural Networkscited by 54
  8. 2018Accelerated Methods for Deep Reinforcement Learningcited by 40
  9. 2015State of the Art Control of Atari Games Using Shallow Reinforcement Learningcited by 23
  10. 2020Importance of using appropriate baselines for evaluation of data-efficiency in deep reinforcement learning for Ataricited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022Human-level Atari 200x fastercited by 28
  2. 2022Value-Consistent Representation Learning for Data-Efficient Reinforcement Learningcited by 15