# [AAFLOW+] Stateful Operator Abstraction with Zero-Copy Distributed KV Cache Orchestration for Multi-Agent Workflows

arXiv: https://arxiv.org/abs/2607.10987 · HTML: https://mlsanity.com/paper/2607.10987/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Efficient reasoning and agent inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Placed in

- Efficient LLM inference and serving / [Efficient reasoning and agent inference](https://mlsanity.com/inference/efficient-reasoning-and-agent-inference/)

## Builds on (most cited first, 22 of 22)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://mlsanity.com/paper/2303.06865/) (2023)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills](https://mlsanity.com/paper/2308.16369/) (2023)
- [MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation](https://mlsanity.com/paper/2409.05591/) (2024)
- [ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition](https://mlsanity.com/paper/2402.15220/) (2024)
- [MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool](https://mlsanity.com/paper/2406.17565/) (2024)
- [KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems](https://mlsanity.com/paper/2510.12872/) (2025)
- [Parsl: Pervasive Parallel Programming in Python](https://mlsanity.com/paper/1905.02158/) (2019)
- [Towards Scalable Dataframe Systems](https://mlsanity.com/paper/2001.00888/) (2020)
- [High Performance Data Engineering Everywhere](https://mlsanity.com/paper/2007.09589/) (2020)
- [Design and Performance Characterization of RADICAL-Pilot on Leadership-class Platforms](https://mlsanity.com/paper/2103.00091/) (2021)
- [Supercharging Distributed Computing Environments For High Performance Data Engineering](https://mlsanity.com/paper/2301.07896/) (2023)
- [Hybrid Cloud and HPC Approach to High-Performance Dataframes](https://mlsanity.com/paper/2212.13732/) (2022)
- [In-depth Analysis On Parallel Processing Patterns for High-Performance Dataframes](https://mlsanity.com/paper/2307.01394/) (2023)
- [Deep RC: A Scalable Data Engineering and Deep Learning Pipeline](https://mlsanity.com/paper/2502.20724/) (2025)
- [AAFLOW: Scalable Patterns for Agentic AI Workflows](https://mlsanity.com/paper/2605.02162/) (2026)
- [Design and Implementation of an Analysis Pipeline for Heterogeneous Data](https://mlsanity.com/paper/2403.15721/) (2024)

## Built on by (most cited first, 2 of 2)

- [Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KV](https://mlsanity.com/paper/2607.23693/) (2026)
- [OpRAG: A Resource-Deterministic Runtime for GPU-Backed Multi-Stage RAG Workflows](https://mlsanity.com/paper/2608.08340/) (2026)

## Abstract (arXiv)

Multi-agent LLM systems increasingly integrate retrieval, planning, and reasoning, but remain fundamentally text-centric, requiring agents to repeatedly recompute shared context through expensive prefill. Although single-request inference is known to be accelerated by KV-cache management, it is usually restricted to local serving scopes. We introduce AAFLOW+, a stateful extension of agentic workflow operators that makes KV cache a first-class distributed systems object. AAFLOW+ builds processes into communication-aware graphs that concurrently optimize data, prompts, and reusable model state. It also provides operators for KV materialization, transfer, fork, composition, and eviction. Its runtime enables zero-copy, transfer-aware execution, allowing agents to reuse long context without recomputation. AAFLOW+ reduces TTFT by up to 50.2x, achieves up to 7.63x reduced multi-agent compute cost at 16-agent scale, reduces KV memory by 1.72-6.10x, and increases throughput by more than 7.74x, based on an analytical cost model parameterized by empirical hardware microbenchmarks. The results demonstrate that KV transmission outperforms recomputation on networks with moderate to high bandwidth, making sure KV-state sharing greatly increases efficiency in multi-agent LLM systems by replacing text passing.