mlsanityData to 14 Oct 2026

Semantic Caching and Intent-Driven Context Optimization for Multi-Agent Natural Language to Code Systems

arXiv 2601.11687 ↗First posted 16 Jan 2026cs.SE cs.AIcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

We present a production-optimized multi-agent system designed to translate natural language queries into executable Python code for structured data analytics. Unlike systems that rely on expensive frontier models, our approach achieves high accuracy and cost efficiency through three key innovations: (1) a semantic caching system with LLM-based equivalence detection and structured adaptation hints that provides cache hit rates of 67% on production queries; (2) a dual-threshold decision mechanism that separates exact-match retrieval from reference-guided generation; and (3) an intent-driven dynamic prompt assembly system that reduces token consumption by 40-60% through table-aware context filtering. The system has been deployed in production for enterprise inventory management, processing over 10,000 queries with an average latency of 8.2 seconds and 94.3% semantic accuracy. We describe the architecture, present empirical results from production deployment, and discuss practical considerations for deploying LLM-based analytics systems at scale.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 37 references that are in the map. First posted on arXiv 16 Jan 2026.

Builds on

17 of 37 references.

References · 17

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  7. 2023StarCoder: may the source be with you!cited by 799
  8. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  9. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  10. 2021Retrieval Augmented Code Generation and Summarizationcited by 149
  11. 2024BEAVER: An Enterprise Benchmark for Text-to-SQLcited by 42
  12. 2024GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Cachingcited by 22
5 more
  1. 2024MeanCache: User-Centric Semantic Caching for LLM Web Servicescited by 14
  2. 2025Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptationcited by 13
  3. 2025Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Datacited by 7
  4. 2025Category-Aware Semantic Caching for Heterogeneous LLM Workloadscited by 4
  5. 2025Text-to-SQL for Enterprise Data Analyticscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ContextPipe: Database-Inspired Context Assembly for Long-Horizon Agents
  2. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation
  3. 2026AgentSM: Semantic Memory for Agentic Text-to-SQL
  4. 2026Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
  5. 2026GenDB: The Next Generation of Query Processing -- Synthesized, Not Engineered
  6. 2025Supporting Dynamic Agentic Workloads: How Data and Agents Interact
  7. 2025Rethinking Query Optimization for Multi-Agent Systems [Vision]
  8. 2026Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agents