mlsanityData to 13 Oct 2026

DORA Explorer: Improving the Exploration Ability of LLMs Without Training

arXiv 2604.17244 ↗First posted 19 Apr 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) agents for sequential decision-making struggle to produce diverse outputs. This leads to insufficient exploration, suboptimal solutions, and repeated actions. Actions are generated at the sequence level, but existing sampling strategies, such as temperature scaling, introduce diversity at the token level, not at the sequence level. We introduce DORA EXPLORER (Diversity-Oriented Ranking of Actions), a training-free, inference-time algorithm for improving exploration in LLM agents. DORA generates multiple candidate actions, scores them using sequence-level log-probability statistics, and samples an action via a tunable exploration parameter. We first study exploration in the classic Multi-Armed Bandit setting, where DORA substantially outperforms temperature-based sampling. Our main evaluation is on the Text Adventure Learning Environment Suite (TALES), where prompting strategies fail to explore but DORA delivers consistent gains across model families, e.g., 31.43% (ReAct) -> 45.5% (DORA) for Qwen-2.5 7B in TextWorld. Beyond exploration, DORA prevents common failures, such as getting stuck in loops. Our project is available at: https://dora-explore.github.io/.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

33 of 42 references.

References · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,656
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,181
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,893
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,823
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,218
  8. 2018Hierarchical Neural Story Generationcited by 1,186
  9. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,167
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,032
  11. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 660
  12. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 477
21 more
  1. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 309
  2. 2018TextWorld: A Learning Environment for Text-based Gamescited by 306
  3. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 170
  4. 2019Interactive Fiction Games: A Colossal Adventurecited by 166
  5. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 148
  6. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 113
  7. 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 102
  8. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 97
  9. 2024Is Temperature the Creativity Parameter of Large Language Models?cited by 92
  10. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 92
  11. 2024Can large language models explore in-context?cited by 75
  12. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 61
  13. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 38
  14. 2024EVOLvE: Evaluating and Optimizing LLMs For In-Context Explorationcited by 34
  15. 2025Training a Generally Curious Agentcited by 30
  16. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  17. 2025Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputscited by 15
  18. 2022TextWorldExpress: Simulating Text Games at One Million Steps Per Secondcited by 13
  19. 2025TALES: Text Adventure Learning Environment Suitecited by 10
  20. 2026Expanding LLM Agent Boundaries with Strategy-Guided Explorationcited by 9
  21. 2018Sentence-wise Smooth Regularization for Sequence to Sequence Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Road Less Traveled: Enhancing Exploration in LLMs via Sequential Sampling
  2. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
  3. 2025Toward Efficient Exploration by Large Language Model Agents
  4. 2025Should You Use Your Large Language Model to Explore or Exploit?
  5. 2026Multi-Agent LLMs for Adaptive Acquisition in Bayesian Optimization
  6. 2025LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models
  7. 2026MAGE: Meta-Reinforcement Learning for Language Agents toward Strategic Exploration and Exploitation
  8. 2025Representation-Based Exploration for Language Models: From Test-Time to Post-Training