QUIVER: A Formal Framework for Quantifying Perturbation Propagation and Bifurcation in Compound AI Systems
Not on the map. Why?
Abstract (arXiv)
Compound AI systems that chain multiple LLM calls into directed computation graphs are now the dominant architecture for production AI. Although these architectures leverage heterogeneous nodes with mixed-mode outputs, no existing framework quantifies how perturbations propagate through such pipelines, where nodes are stochastic and execution paths can diverge structurally. We introduce QUIVER, a formal framework for measuring perturbation propagation in graph-structured LLM pipelines. The framework defines: (1) a sensitivity matrix with type-dispatched distance metrics that classifies edges as amplifiers, absorbers, or threshold-sensitive, complemented by occurrence-lift; (2) trajectory divergence decomposing variation into value drift, structural path divergence, and iteration count divergence; (3) bifurcation thresholds identifying the smallest perturbation that causes structural execution path changes; and (4) distribution faithfulness, quantifying when per node evaluation datasets diverge from production distributions. We validate on two production enterprise pipelines and a public DSPy multihop QA pipeline, three structurally distinct architectures. Across 8,200+ instrumented traces (32,000+ pair comparisons), we demonstrate that QUIVER reveals distinct sensitivity profiles across architectures, distinguishes mechanistically different cascade patterns producing identical divergence rates, predicts nodes prone to trajectory bifurcation from observational data alone, and localizes stale evaluation artifacts to specific node-field categories that aggregate metrics cannot surface.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 24 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
22 of 24 references.
References · 22
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
- 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
- 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
- 2024TextGrad: Automatic "Differentiation" via Textcited by 273
- 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 154
- 2025Towards a Science of Scaling Agent Systemscited by 138
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 116
- 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
- 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 43
- 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Makingcited by 42
10 more
- 2025LLM-AutoDiff: Auto-Differentiate Any LLM Workflowcited by 21
- 2025RAFFLES: Reasoning-based Attribution of Faults for LLM Systemscited by 21
- 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 14
- 2025An Information Theoretic Perspective on Agentic System Designcited by 8
- 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
- 2025Geometric Dynamics of Agentic Loops in Large Language Modelscited by 4
- 2026ValueFlow: Measuring the Propagation of Value Perturbations in Multi-Agent LLM Systemscited by 4
- 2026A Control-Theoretic Foundation for Agentic Systemscited by 4
- 2026TextResNet: Decoupling and Routing Optimization Signals in Compound AI Systems via Deep Residual Tuningcited by 3
- 2026Textual Equilibrium Propagation for Deep Compound AI Systemscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.