mlsanityData to 7 Oct 2026

Randomized Prior Functions for Deep Reinforcement Learning

arXiv 1806.03335 ↗First posted 8 Jun 2018stat.ML cs.AI cs.LGcited by 218

Not on the map. Why?

Cited by 26 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Dealing with uncertainty is essential for efficient reinforcement learning. There is a growing literature on uncertainty estimation for deep learning from fixed datasets, but many of the most popular approaches are poorly-suited to sequential decision problems. Other methods, such as bootstrap sampling, have no mechanism for uncertainty that does not come from the observed data. We highlight why this can be a crucial shortcoming and propose a simple remedy through addition of a randomized untrainable `prior' network to each ensemble member. We prove that this approach is efficient with linear representations, provide simple illustrations of its efficacy with nonlinear representations and show that this approach scales to large-scale problems far better than previous attempts.

Key facts. Cited by 218 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 41; 2022: 49; 2023: 46; 2024: 24; 2025: 27; 2026: 30.

218 papers.

Sort:

By area:

  1. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  2. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  3. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  4. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  5. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  6. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
  7. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  8. 2021Epistemic Neural Networkscited by 120
  9. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  10. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  11. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
  12. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 78
38 more
  1. 2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 72
  2. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  3. 2021Deep Graph-level Anomaly Detection by Glocal Knowledge Distillationcited by 67
  4. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 66
  5. 2021Reinforcement Learning, Bit by Bitcited by 59
  6. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  7. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 56
  8. 2022Uncertainty-guided Source-free Domain Adaptationcited by 46
  9. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  10. 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
  11. 2021Learning more skills through optimistic explorationcited by 37
  12. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 36
  13. 2022Uncertainty Estimation for Language Reward Modelscited by 35
  14. 2021Geometric Entropic Explorationcited by 32
  15. 2023Approximate Thompson Sampling via Epistemic Neural Networkscited by 31
  16. 2023Thompson sampling for improved exploration in GFlowNetscited by 31
  17. 2021Ensemble Bootstrapping for Q-Learningcited by 30
  18. 2022A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learningcited by 30
  19. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  20. 2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
  21. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  22. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
  23. 2024Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learningcited by 27
  24. 2021Principled Exploration via Optimistic Bootstrapping and Backward Inductioncited by 26
  25. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coveragecited by 26
  26. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 26
  27. 2022An Analysis of Ensemble Samplingcited by 23
  28. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 23
  29. 2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
  30. 2021The Neural Testbed: Evaluating Joint Predictionscited by 22
  31. 2022Fine-Tuning Language Models via Epistemic Neural Networkscited by 22
  32. 2022Bayesian Learning for Neural Networks: an algorithmic surveycited by 22
  33. 2021Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networkscited by 21
  34. 2022Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Sizecited by 20
  35. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20
  36. 2022Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learningcited by 19
  37. 2022Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrappingcited by 19
  38. 2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18

50 most cited of 218 All 218 (JSON)