# Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States

arXiv: https://arxiv.org/abs/2610.01415 · HTML: https://mlsanity.com/paper/2610.01415/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 26 of 26)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024)
- [ACON: Optimizing Context Compression for Long-horizon LLM Agents](https://mlsanity.com/paper/2510.00615/) (2025)
- [WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment](https://mlsanity.com/paper/2402.12275/) (2024)
- [HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model](https://mlsanity.com/paper/2408.09559/) (2024)
- [AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents](https://mlsanity.com/paper/2407.04363/) (2024)
- [Context as a Tool: Context Management for Long-Horizon SWE-Agents](https://mlsanity.com/paper/2512.22087/) (2025)
- [(How) Do Language Models Track State?](https://mlsanity.com/paper/2503.02854/) (2025)
- [ReflAct: World-Grounded Decision Making in LLM Agents via Goal-State Reflection](https://mlsanity.com/paper/2505.15182/) (2025)
- [StateAct: Enhancing LLM Base Agents via Self-prompting and State-tracking](https://mlsanity.com/paper/2410.02810/) (2024)
- [COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context](https://mlsanity.com/paper/2510.08790/) (2025)
- [Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management](https://mlsanity.com/paper/2508.04664/) (2025)
- [From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models](https://mlsanity.com/paper/2504.06214/) (2025)
- [LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth](https://mlsanity.com/paper/2602.07962/) (2026)
- [ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction](https://mlsanity.com/paper/2512.20111/) (2025)
- [The Context Gathering Decision Process: A POMDP Framework for Agentic Search](https://mlsanity.com/paper/2605.07042/) (2026)
- [Graph of States: Solving Abductive Tasks with Large Language Models](https://mlsanity.com/paper/2603.21250/) (2026)
- [Tru-POMDP: Task Planning Under Uncertainty via Tree of Hypotheses and Open-Ended POMDPs](https://mlsanity.com/paper/2506.02860/) (2025)
- [Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models](https://mlsanity.com/paper/2511.23319/) (2025)
- [A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis](https://mlsanity.com/paper/2606.29193/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents can now undertake increasingly complex tasks, but the way they organize interaction history into memory does not ensure a coherent understanding of the current world. We introduce PoS, an inference-time framework that constructs and continually maintains explicit belief states as the agent's decision context. Each belief combines an estimate of the current world state with unresolved task requirements, making explicit what the agent still needs to learn and accomplish. To keep this belief reliable and actionable, PoS validates its consistency and monitors task progress to detect Belief Trapping, where the agent continues to act without making meaningful progress toward the goal. Recovery is then tailored to both the trapping pattern and the type of unresolved task requirement. Experiments on four benchmarks spanning execution and diagnosis show that PoS achieves the highest overall performance on every benchmark with all three LLM backbones. Ablations demonstrate the importance of consistency validation and recovery, while context-scaling experiments show resilience to context growth. Together, these results support belief construction and continual maintenance as a foundation for long-horizon context management beyond history retention and compression.