mlsanityData to 14 Oct 2026

When to ASK: Uncertainty-Gated Language Assistance for Reinforcement Learning

arXiv 2604.02226 ↗First posted 2 Apr 2026cs.AI cs.LGcited by 2

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Reinforcement learning (RL) agents often struggle with out-of-distribution (OOD) scenarios, leading to high uncertainty and random behavior. While language models (LMs) contain valuable world knowledge, larger ones incur high computational costs, hindering real-time use, and exhibit limitations in autonomous planning. We introduce Adaptive Safety through Knowledge (ASK), which combines smaller LMs with trained RL policies to enhance OOD generalization without retraining. ASK employs Monte Carlo Dropout to assess uncertainty and queries the LM for action suggestions only when uncertainty exceeds a set threshold. This selective use preserves the efficiency of existing policies while leveraging the language model's reasoning in uncertain situations. In experiments on the FrozenLake environment, ASK shows no improvement in-domain, but demonstrates robust navigation in transfer tasks, achieving a reward of 0.95. Our findings indicate that effective neuro-symbolic integration requires careful orchestration rather than simple combination, highlighting the need for sufficient model scale and effective hybridization mechanisms for successful OOD generalization.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 24 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

12 of 24 references.

References · 12

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  2. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  3. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  5. 2017Safe Reinforcement Learning via Shieldingcited by 449
  6. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 312
  7. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  8. 2020Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learningcited by 16
  9. 2020The Hidden Uncertainty in a Neural Networks Activationscited by 14
  10. 2025How Far Are LLMs from Symbolic Planners? An NLP-Based Perspectivecited by 6
  11. 2024Safe Learning of PDDL Domains with Conditional Effects -- Extended Versioncited by 2
  12. 2025Retrieval-augmented reasoning with lean language modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026When in Doubt, Plan It Out: Committed Small Language Model Deliberation for Reactive Reinforcement Learningcited by 1
  2. 2026ASK in the Dark: Uncertainty-Gated LLM Assistance under Partial Observabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Guiding Reinforcement Learning Using Uncertainty-Aware Large Language Models
  2. 2024Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
  3. 2025Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning
  4. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
  5. 2026PriorZero: Bridging Language Priors and World Models for Decision Making
  6. 2025Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs
  7. 2025Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
  8. 2026Uncertainty-Aware LLM-Guided Policy Shaping for Sparse-Reward Reinforcement Learning