When Does Memory Help Multi-Trajectory Inference for Tool-Use LLM Agents?
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Multi-trajectory inference for tool-use LLM agents - generating multiple reasoning attempts and selecting among them - benefits from transferring knowledge across attempts so that later ones avoid the pitfalls of earlier ones. Existing cross-trajectory memory methods (trajectory-level reflection, atomic fact extraction, raw observation injection) are each evaluated under a single inference strategy on a single task, making it unclear whether reported gains reflect properties of the memory abstraction or of the inference method. We propose a unified framework that decomposes memory along two axes -- the scope of transfer (within an expansion vs. across trajectories) and the abstraction of the transferred content -- and evaluate four methods under three inference strategies (best-of-N, beam search, MCTS) on four tool-use benchmarks spanning SQL, knowledge-graph, and CLI environments, in a verifier-free setting that matches the deployment regime of practical agents. The experiment matrix identifies the inference method as a confound: the same memory method produces statistically distinct results under different inference strategies on the same examples. Reflection reaches significance only under MCTS (not under best-of-N); within-expansion injection (conditioning each candidate on prior siblings' outcomes) helps only diversity-starved beam search; and atomic fact extraction is accuracy-neutral but shortens trajectories by 19-26% on tasks with reusable environmental structure.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 15 of its 24 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
15 of 24 references.
Closest ancestors · 3
Same direction, newest first
- 2025SwarmSys: Decentralized Swarm-Inspired Agents for Scalable and Adaptive Reasoning11 Oct
- 2024ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning2 Oct
- 2023Reasoning with Language Model is Planning with World Model24 May
Other references · 12
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2017Hindsight Experience Replaycited by 1,136
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
- 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 121
- 2024INMS: Memory Sharing for Large Language Model based Agentscited by 31
- 2025Fact-Augmented Lookahead Planning for LLM Agentscited by 8
- 2026MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillationcited by 7
- 2025Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?cited by 5
- 2026LiTS: A Modular Framework for LLM Tree Searchcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026When Does Overlap Help? OSU-Mem and a Cell-Conditional Analysis of Trajectory Memory for LLM Agents
- 2025Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents
- 2026Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents
- 2026Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell
- 2026Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents
- 2026Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline
- 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
- 2026Akashic: A Low-Overhead LLM Inference Service with MemAttention