Data to 7 Oct 2026

Generalization and Exploration via Randomized Value Functions

arXiv 1402.0635 ↗First posted 4 Feb 2014stat.ML cs.AI cs.LGcited by 112

Not on the map. Why?

Abstract (arXiv)

We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly inefficient, and we present computational results that demonstrate dramatic efficiency gains enjoyed by RLSVI. Further, we establish an upper bound on the expected regret of RLSVI that demonstrates near-optimality in a tabula rasa learning context. More broadly, our results suggest that randomized value functions offer a promising approach to tackling a critical challenge in reinforcement learning: synthesizing efficient exploration and effective generalization.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 25; 2023: 21; 2024: 15; 2025: 17; 2026: 11.

112 papers.

Sort:

By area:

2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
2021Epistemic Neural Networkscited by 120
2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
2021Randomized Exploration for Reinforcement Learning with General Value Function Approximationcited by 49
2023Making RL with Preference-based Feedback Efficient via Randomizationcited by 46
2022Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexitycited by 31
2023Thompson sampling for improved exploration in GFlowNetscited by 31
2021Minimax Regret for Stochastic Shortest Pathcited by 30
2022A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learningcited by 30
2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30

38 more

2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
2021Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisitingcited by 27
2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 22
2021Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learningcited by 21
2021Cross-Trajectory Representation Learning for Zero-Shot Generalization in RLcited by 20
2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20
2022Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learningcited by 19
2022Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learningcited by 18
2025Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learningcited by 17
2022Learn to Match with No Regret: Reinforcement Learning in Markov Matching Marketscited by 16
2023Posterior Sampling for Deep Reinforcement Learningcited by 16
2021The Value of Information When Deciding What to Learncited by 15
2022Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiencycited by 15
2025Toward Efficient Exploration by Large Language Model Agentscited by 15
2021Near-Optimal Randomized Exploration for Tabular Markov Decision Processescited by 14
2022A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Gamescited by 14
2021An L2 Analysis of Reinforcement Learning in High Dimensions with Kernel and Neural Network Approximationcited by 13
2022Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networkscited by 13
2022Model-Based Reinforcement Learning with Multinomial Logistic Function Approximationcited by 13
2023Exploration via linearly perturbed loss minimisationcited by 13
2022Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guaranteescited by 12
2022Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planningcited by 12
2023Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounderscited by 12
2024More Efficient Randomized Exploration for Reinforcement Learning via Approximate Samplingcited by 12
2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 11
2021Safe Policy Optimization with Local Generalized Linear Function Approximationscited by 10
2022Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Samplingcited by 10
2022Exploring through Random Curiosity with General Value Functionscited by 10
2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
2024Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedbackcited by 10
2021Rule-Based Reinforcement Learning for Efficient Robot Navigation with Space Reductioncited by 9
2022Foundation Models for Semantic Novelty in Reinforcement Learningcited by 9
2022Improved Regret for Differentially Private Exploration in Linear MDPcited by 8
2023Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Modelscited by 8
2024A Survey Analyzing Generalization in Deep Reinforcement Learningcited by 8
2025Exploration-Driven Generative Interactive Environmentscited by 8
2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8

50 most cited of 112 All 112 (JSON)