mlsanityData to 14 Oct 2026

Flexible Empowerment at Reasoning with Extended Best-of-N Sampling

arXiv 2604.15614 ↗First posted 17 Apr 2026cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel method that incorporates empowerment when reasoning actions in reinforcement learning (RL), thereby achieving the flexibility of exploration-exploitation dilemma (EED). In previous methods, empowerment for promoting exploration has been provided as a bonus term to the task-specific reward function as an intrinsically-motivated RL. However, this approach introduces a delay until the policy that accounts for empowerment is learned, making it difficult to adjust the emphasis on exploration as needed. On the other hand, a trick devised for fine-tuning recent foundation models at reasoning, so-called best-of-N (BoN) sampling, allows for the implicit acquisition of modified policies without explicitly learning them. It is expected that applying this trick to exploration-promoting terms, such as empowerment, will enable more flexible adjustment of EED. Therefore, this paper investigates BoN sampling for empowerment. Furthermore, to adjust the degree of policy modification in a generalizable manner while maintaining computational cost, this paper proposes a novel BoN sampling method extended by Tsalis statistics. Through toy problems, the proposed method's cability to balance EED is verified. In addition, it is demonstrated that the proposed method improves RL performance to solve complex locomotion tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

31 of 47 references.

References · 31

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  4. 2019Root Mean Square Layer Normalizationcited by 1,480
  5. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  6. 2018Exploration by Random Network Distillationcited by 993
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  9. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  10. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  11. 2019Self-Supervised Exploration via Disagreementcited by 265
  12. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
19 more
  1. 2019Sparse Sequence-to-Sequence Modelscited by 170
  2. 2017Parameter Space Noise for Explorationcited by 161
  3. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  4. 2017Boltzmann Exploration Done Rightcited by 69
  5. 2023How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligencecited by 58
  6. 2025Evaluation of Best-of-N Sampling Strategies for Language Model Alignmentcited by 39
  7. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  8. 2025Soft Best-of-n Sampling for Model Alignmentcited by 30
  9. 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
  10. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  11. 2023Latent Exploration for Reinforcement Learningcited by 22
  12. 2021Squareplus: A Softplus-Like Algebraic Rectifiercited by 19
  13. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  14. 2025AdaSplash: Adaptive Sparse Flash Attentioncited by 12
  15. 2022AdaTerm: Adaptive T-Distribution Estimated Robust Moments for Noise-Robust Stochastic Gradient Optimizationcited by 11
  16. 2022Consolidated Adaptive T-soft Update for Deep Reinforcement Learningcited by 5
  17. 2022Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Searchcited by 5
  18. 2024Surprise-Adaptive Intrinsic Motivation for Unsupervised Reinforcement Learningcited by 4
  19. 2025Information-Theoretic Policy Pre-Training with Empowermentcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Redesigning Regularization for Effective Policy Smoothingcited by 0
  2. 2026Shattering the Autoregressive Curse: Dynamic Epistemic Entropy Orchestrated Erasable Reinforcement Learning for LLMscited by 0
  3. 2026ExpBoN: Exponential-Noise Best-of-n for Efficient Test-Time LLM Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RL
  2. 2025Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewards
  3. 2026Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data
  4. 2026Exploration-Preserving Policy Optimization
  5. 2026Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilities
  6. 2025The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation
  7. 2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning
  8. 2026Reasoning with Sampling: Cutting at Decision Points