mlsanityData to 7 Oct 2026

Accelerating Language Model Workflows with Prompt Choreography

arXiv 2512.23049 ↗First posted 28 Dec 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are increasingly deployed in multi-agent workflows. We introduce Prompt Choreography, a framework that efficiently executes LLM workflows by maintaining a dynamic, global KV cache. Each LLM call can attend to an arbitrary, reordered subset of previously encoded messages. Parallel calls are supported. Though caching messages' encodings sometimes gives different results from re-encoding them in a new context, we show in diverse settings that fine-tuning the LLM to work with the cache can help it mimic the original results. Prompt Choreography significantly reduces per-message latency (2.0--6.2× faster time-to-first-token) and achieves substantial end-to-end speedups (>2.2×) in some workflows dominated by redundant computation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 47 references that are in the map. First posted on arXiv 28 Dec 2025.

Builds on

32 of 47 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  10. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
20 more
  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  4. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  5. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  6. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  7. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  8. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  9. 2019MLPerf Inference Benchmarkcited by 195
  10. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  11. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  12. 2022Language Model Cascadescited by 104
  13. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 104
  14. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partitioncited by 81
  15. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  16. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 60
  17. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 55
  18. 2024DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Servingcited by 30
  19. 2024KBLaM: Knowledge Base augmented Language Modelcited by 16
  20. 2024Do Large Language Models Need a Content Delivery Network?cited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
  2. 2026Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
  3. 2025PromptBridge: Cross-Model Prompt Transfer for Large Language Models
  4. 2026Characterizing Large Language Model Agentic Workflows: A Study on N8n Ecosystem
  5. 2024APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts
  6. 2026TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
  7. 2024AutoFlow: Automated Workflow Generation for Large Language Model Agents
  8. 2026Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflows