mlsanityData to 7 Oct 2026

AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents

arXiv 2607.02255 ↗First posted 2 Jul 2026cs.AI cs.CLcited by 1

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Memory for a long-horizon LLM agent is a contract about what each future decision is allowed to see. The simplest contract appends past observations, tool calls, and reflections to every prompt, which makes prior context easy to access but also turns it into a jumbled mixture in which the effect of any single memory component is hard to isolate. We introduce and instrument an alternative bounded contract: every decision is made from a fresh user message assembled by typed retrieval, with no raw cross-decision transcript appended. The prompt thus stays bounded across runs of any length, and any single layer can be ablated in isolation. We instantiate the contract in Slay the Spire 2, a closed-rule stochastic deck-building game whose runs require hundreds of tactical and strategic decisions. A public online benchmark of frontier LLMs on the same game reports zero wins at the lowest difficulty across five configurations, and the developer-reported human win rate at the same difficulty is 16%; the task is hard but not saturated. Within our harness, a fixed-A0 ablation shows the largest observed difference when triggered strategic skills are enabled: the no-store baseline wins 3/10 games and adding the skill layer 6/10. At this sample size the comparison is directional rather than statistically decisive (Fisher exact p≈0.37); a cross-backbone probe and public accumulating-context baselines are reported as operational comparisons rather than controlled tests of the contract variable itself. We release a reproducible testbed: 298 completed trajectories with condition tags, frozen memory/skill snapshots, prompt records, and analysis scripts -- an agent design and a validated, reusable methodology for studying how explicit memory layers shape long-horizon LLM-agent decisions.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 46 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

16 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environments8 Mar
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 13

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  4. 2025Memory in the Age of AI Agentscited by 285
  5. 2020The NetHack Learning Environmentcited by 196
  6. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  7. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
  8. 2026Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agentscited by 22
  9. 2026Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Taskscited by 18
  10. 2025Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinementcited by 17
  11. 2023Mastering Strategy Card Game (Hearthstone) with Improved Techniquescited by 10
  12. 2025DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learningcited by 7
1 more
  1. 2025UrzaGPT: LoRA-Tuned Large Language Models for Card Selection in Collectible Card Gamescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Improving Sequential Decision-Making in LLM Agents via Experience Memory
  2. 2026LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning
  3. 2026The Epistemics of Agent Memory: Measuring, and Governing, the Consolidation Decision in Long-Horizon LLM Agents
  4. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  5. 2026MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games
  6. 2026ElasticMem: Latent Memory as a Learnable Resource for LLM Agents
  7. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
  8. 2025Memory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Games