# DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents

arXiv: https://arxiv.org/abs/2608.20664 · HTML: https://mlsanity.com/paper/2608.20664/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 30 of 32)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?](https://mlsanity.com/paper/2509.16941/) (2025)
- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks](https://mlsanity.com/paper/2602.16313/) (2026)
- [MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling](https://mlsanity.com/paper/2511.11793/) (2025)
- [A Self-Improving Coding Agent](https://mlsanity.com/paper/2504.15228/) (2025)
- [Tongyi DeepResearch Technical Report](https://mlsanity.com/paper/2510.24701/) (2025)
- [Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs](https://mlsanity.com/paper/2510.27246/) (2025)
- [AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications](https://mlsanity.com/paper/2602.22769/) (2026)
- [Chain-of-Retrieval Augmented Generation](https://mlsanity.com/paper/2501.14342/) (2025)
- [Sleep-time Compute: Beyond Inference Scaling at Test-time](https://mlsanity.com/paper/2504.13171/) (2025)
- [MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verification](https://mlsanity.com/paper/2603.15726/) (2026)
- [Useful Memories Become Faulty When Continuously Updated by LLMs](https://mlsanity.com/paper/2605.12978/) (2026)
- [EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective](https://mlsanity.com/paper/2605.18421/) (2026)
- [MiroFlow: Towards High-Performance and Robust Open-Source Agent Framework for General Deep Research Tasks](https://mlsanity.com/paper/2602.22808/) (2026)
- [Are We Ready For An Agent-Native Memory System?](https://mlsanity.com/paper/2606.24775/) (2026)
- [Structurally Aligned Subtask-Level Memory for Software Engineering Agents](https://mlsanity.com/paper/2602.21611/) (2026)
- [Beyond Similarity: Trustworthy Memory Search for Personal AI Agents](https://mlsanity.com/paper/2606.06054/) (2026)
- [WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction](https://mlsanity.com/paper/2605.29341/) (2026)

## Abstract (arXiv)

DreamBench-SWE is a multi-session benchmark for software-agent memory hygiene in which later software tasks depend on non-inferable evidence from earlier sessions and are scored by executable hidden oracles. We report the original scaled v2 fold and a separately preregistered v2.1 successor audit designed after that study but frozen before successor outcome inspection. The successor run completed 360/360 work units and 720/720 S3 cells across four conditions. In the original fold, the primary DF-hybrid--B5 contrast was null (95/180 versus 89/180; clustered p=.518, Holm p=1), not evidence of equivalence, and C9/C10 retained B0-headroom limitations. In the successor, no external memory achieved 21/180 passes (rate 0.1167), deterministic verbatim event memory 82/180 (rate 0.4556), the typed-plus-raw reference probe 83/180 (rate 0.4611), and one pinned hosted Mem0 literal-storage configuration 97/180 (rate 0.5389). The registered six-slot Family A retained unavailable slots at p=1; all three available comparisons against no memory rejected after Holm correction. Both preregistered mechanism contrasts were unavailable after pre-evaluation conformance rejection. The secondary literal-storage-versus-verbatim comparison was nonconfirmatory and sensitivity-dependent, while the comparison with the reference probe did not reject. The audit therefore supports DreamBench-SWE as a discriminating executable profile benchmark and characterizes one exact hosted-memory configuration, but it does not establish an external-system mechanism, superiority among memory-bearing conditions, equivalence, or broad product generality. The original v2.0.5 findings and artifacts remain unchanged.