mlsanityData to 7 Oct 2026

Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs

arXiv 2607.03426 ↗First posted 3 Jul 2026cs.LG cs.AIcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large language models (LLMs) exhibit strong reasoning and world-knowledge capabilities, yet often struggle to gather information effectively across the multi-turn interactions required in sequential decision-making settings. We introduce Amortised Sequential Information Gathering (ASIG), a fine-tuning approach that amortises Bayesian Experimental Design (BED) into LLM policies via a multi-turn extension of Group Relative Policy Optimisation with an Expected Information Gain reward. Evaluated on the 20 Questions task, ASIG more than doubles the success rate of the 7B base model and reduces inference cost by over 25× relative to BED-LLM, a competitive inference-time baseline. Applied to MediQ, a medical diagnosis benchmark unseen during training, ASIG improves information-seeking performance at the 7B scale, suggesting that the learned strategies can transfer out of distribution. Our findings show that amortising BED into LLM policies provides an effective and computationally efficient approach to sequential information gathering.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 33 references that are in the map. First posted on arXiv 3 Jul 2026.

Builds on

24 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 22

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2020Aligning AI With Shared Human Valuescited by 706
  8. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  9. 2024Iterative Reasoning Preference Optimizationcited by 244
  10. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 145
  11. 2023Modern Bayesian Experimental Designcited by 121
  12. 2017On Nesting Monte Carlo Estimatorscited by 82
10 more
  1. 2019A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experimentscited by 58
  2. 2025Training a Generally Curious Agentcited by 29
  3. 2023Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Gamescited by 24
  4. 2025Active Task Disambiguation with LLMscited by 23
  5. 2024Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gaincited by 21
  6. 2025BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Designcited by 21
  7. 2024CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrievalcited by 18
  8. 2026Intrinsic Credit Assignment for Long Horizon Interactioncited by 9
  9. 2025Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like Peoplecited by 7
  10. 2025Sherlock Your Queries: Learning to Ask the Right Questions for Dialogue-Based Retrievalcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world modelscited by 6
  2. 2026HPOQuest: A Rare-Disease Diagnostic Agent Using Active Phenotype Acquisitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Amortizing intractable inference in large language models
  2. 2026CA-BED: Conversation-Aware Bayesian Experimental Design
  3. 2025Revisiting LLM Reasoning via Information Bottleneck
  4. 2026Bayesian Orchestration of Multi-LLM Agents for Cost-Aware Sequential Decision-Making
  5. 2025LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?
  6. 2025Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs
  7. 2025seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs
  8. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey