# A hierarchical memory architecture overcomes context limits in long-horizon multi-agent computational modeling

arXiv: https://arxiv.org/abs/2607.07666 · HTML: https://mlsanity.com/paper/2607.07666/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 7 of its 33 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 7 of 7)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Sources of Hallucination by Large Language Models on Inference Tasks](https://mlsanity.com/paper/2305.14552/) (2023)

## Abstract (arXiv)

Large language models (LLMs) demonstrate remarkable reasoning capabilities, yet their stateless architecture fundamentally limits deployment in long-horizon research workflows requiring multi-session continuity and quantitative rigor. Here we present Ensemble QSP, a multi-agent framework featuring a three-layer hierarchical memory architecture that bounds injected context (median 301 tokens, max 4,050) by capping state categories and evicting completed work. This enables continuous autonomous operation without context degradation. The system orchestrates five specialist worker agents under domain-expert principal investigators (PIs), enforcing physical constraints through physics-based checklists and structured domain knowledge. Comprehensive benchmarking demonstrates autonomous pharmacokinetic-pharmacodynamic (PKPD) model selection, improved parameter recovery relative to single-agent baselines, and robust interpretation of linguistically diverse prompts. Replication with open-weight models (DeepSeek-V4-Flash/Pro, Llama 3.1 70B) confirmed these architectural conclusions across PKPD modeling, literature synthesis, and PBPK model implementation, proving the framework is independent of proprietary LLMs. Feature-level ablations show that memory, retrieval, and PI oversight address distinct scientific failure modes, though underlying LLM capability remains consequential for stringent physical-consistency checks. The architecture is structurally agnostic to computational biology; adding a new scientific domain requires only a new PI-agent configuration.