# SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems

arXiv: https://arxiv.org/abs/2601.16286 · HTML: https://mlsanity.com/paper/2601.16286/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 22 Jan 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 10 of 10)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents](https://mlsanity.com/paper/2506.14852/) (2025)
- [SCALM: Towards Semantic Caching for Automated Chat Services with Large Language Models](https://mlsanity.com/paper/2406.00025/) (2024)
- [SiriusBI: A Comprehensive LLM-Powered Solution for Data Analytics in Business Intelligence](https://mlsanity.com/paper/2411.06102/) (2024)
- [InstCache: A Predictive Cache for LLM Serving](https://mlsanity.com/paper/2411.13820/) (2024)
- [Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching](https://mlsanity.com/paper/2509.17360/) (2025)
- [Exploring Language Patterns of Prompts in Text-to-Image Generation and Their Impact on Visual Diversity](https://mlsanity.com/paper/2504.14125/) (2025)
- [An Ensemble Embedding Approach for Improving Semantic Caching Performance in LLM-based Systems](https://mlsanity.com/paper/2507.07061/) (2025)

## Built on by (most cited first, 4 of 4)

- [Cross-Domain Uncertainty Quantification for Selective Prediction: A Comprehensive Bound Ablation with Transfer-Informed Betting](https://mlsanity.com/paper/2603.08907/) (2026)
- [Why Agent Caching Fails and How to Fix It: Structured Intent Canonicalization with Few-Shot Learning](https://mlsanity.com/paper/2602.18922/) (2026)
- [AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent](https://mlsanity.com/paper/2604.06296/) (2026)
- [Workload-Aware Caching for Multi-Agent Systems](https://mlsanity.com/paper/2607.20495/) (2026)

## Abstract (arXiv)

Agentic AI pipelines suffer from a hidden inefficiency: they frequently reconstruct identical intermediate logic, such as metric normalization or chart scaffolding, even when the user's natural language phrasing is entirely novel. Conventional boundary caching fails to capture this inefficiency because it treats inference as a monolithic black box. We introduce SemanticALLI, a pipeline-aware architecture within Alli (PMG's marketing intelligence platform), designed to operationalize redundant reasoning. By decomposing generation into Analytic Intent Resolution (AIR) and Visualization Synthesis (VS), SemanticALLI elevates structured intermediate representations (IRs) to first-class, cacheable artifacts. The impact of caching within the agentic loop is substantial. In our evaluation, baseline monolithic caching caps at a 38.7% hit rate due to linguistic variance. In contrast, our structured approach allows for an additional stage, the Visualization Synthesis stage, to achieve an 83.10% hit rate, bypassing 4,023 LLM calls with a median latency of just 2.66 ms. This internal reuse reduces total token consumption, offering a practical lesson for AI system design: even when users rarely repeat themselves, the pipeline often does, at stable, structured checkpoints where caching is most reliable.