mlsanityData to 7 Oct 2026

Align While Search: Belief-Guided Exploratory Inference for World-Grounded Embodied Agents

arXiv 2512.24461 ↗First posted 30 Dec 2025cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

In this paper, we propose a test-time adaptive agent that performs exploratory inference through posterior-guided belief refinement without relying on gradient-based updates or additional training for LLM agent operating under partial observability. Our agent maintains an external structured belief over the environment state, iteratively updates it via action-conditioned observations, and selects actions by maximizing predicted information gain over the belief space. We estimate information gain using a lightweight LLM-based surrogate and assess world alignment through a novel reward that quantifies the consistency between posterior belief and ground-truth environment configuration. Experiments show that our method outperforms inference-time scaling baselines such as prompt-augmented or retrieval-enhanced LLMs, in aligning with latent world states with significantly lower integration overhead.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 45 references that are in the map. First posted on arXiv 30 Dec 2025.

Builds on

28 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025World Model Implanting for Test-time Adaptation of Embodied Agents4 Sep
  2. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models8 Dec

Other references · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2024GPT-4o System Cardcited by 4,467
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  12. 2023Reasoning with Language Model is Planning with World Modelcited by 846
14 more
  1. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  2. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  3. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  4. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  5. 2024Agent Planning with World Knowledge Modelcited by 66
  6. 2023SayCanPay: Heuristic Planning with Large Language Models using Learnable Domain Knowledgecited by 49
  7. 2025MPO: Boosting LLM Agents with Meta Plan Optimizationcited by 36
  8. 2025ReflAct: World-Grounded Decision Making in LLM Agents via Goal-State Reflectioncited by 30
  9. 2023Modeling Dynamic Environments with Scene Graph Memorycited by 18
  10. 2021Towards Optimal Correlational Object Searchcited by 14
  11. 2025Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM Reasoningcited by 8
  12. 2025Enhancing Decision-Making of Large Language Models via Actor-Criticcited by 7
  13. 2022A Contextual Bandit Approach for Learning to Plan in Environments with Probabilistic Goal Configurationscited by 4
  14. 2025Reflect-then-Plan: Offline Model-Based Planning through a Doubly Bayesian Lenscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents
  2. 2026Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
  3. 2026Test-Time Mixture of World Models for Embodied Agents in Dynamic Environments
  4. 2026HarnessWAM: Bridging Prediction and Deliberation in World Action Models
  5. 2025Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response
  6. 2026Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty
  7. 2026Ask the World Before Acting: Environment Probing for Calibrated Agent World Models
  8. 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning