Cost-Governed RAG: Unified Per-Tenant Cost Attribution Across Retrieval and Generation in Multi-Tenant LLM Systems
On the map: Retrieval-augmented generation → RAG pipeline design and evaluation
Abstract (arXiv)
Enterprise Retrieval-Augmented Generation (RAG) deployments face a critical governance gap: while LLM generation cost is metered per token, the retrieval layer - vector memory, similarity compute, and embedding API calls - remains an unattributed shared cost, enabling invisible cross-subsidization among tenants. We present Cost-Governed RAG, an architecture that integrates a codebook-oblivious vector index (TurboVec) with a multi-tenant LLM governance gateway, creating a unified observability stack where embedding, retrieval, and generation costs are jointly attributable per tenant. The architecture exploits TurboVec's deterministic, closed-form memory formula to enable near-exact per-tenant retrieval cost calculation - a property unavailable in graph-based indexes with non-linear memory overhead. Deployed on Snowpark Container Services within a cloud data platform's governance boundary, the system achieves 99.96% end-to-end cost attribution accuracy across 100 simulated tenants (10M vectors, log-normal size distribution) with telemetry overhead below 0.04% of query latency. The architecture reduces retrieval infrastructure cost by 3.1-9.0x compared to managed vector database services under the pricing assumptions detailed in Section IV. We formalize a three-layer cost model and demonstrate that codebook-oblivious quantization enables deterministic per-tenant cost attribution while also removing the shared-codebook leakage surface present in trained quantizers - the latter observation being exploratory and subject to the limitations described in Section VII.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 25 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
11 of 25 references.
Closest ancestors · 2
Same direction, newest first
- 2025RAG-Stack: Co-Optimizing RAG Quality and Performance From the Vector Database Perspective23 Oct
- 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving18 Mar
Other references · 9
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2019Green AIcited by 627
- 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
- 2024RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Searchcited by 48
- 2025ProCut: LLM Prompt Compression via Attribution Estimationcited by 6
- 2026UCCI: Calibrated Uncertainty for Cost-Optimal LLM Cascade Routingcited by 4
- 2026OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Servicescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
- 2026TurboVec: A Case Study in Cost-Efficient Private Retrieval for Enterprise RAG via Codebook-Oblivious Quantization
- 2026Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs
- 2025Engineering the RAG Stack: A Comprehensive Review of the Architecture and Trust Frameworks for Retrieval-Augmented Generation Systems
- 2026EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval
- 2024EdgeRAG: Online-Indexed RAG for Edge Devices
- 2026RAGPerf: An End-to-End Benchmarking Framework for Retrieval-Augmented Generation Systems
- 2026RAGe: A Retrieval-Augmented Generation Evaluation Framework