# Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agents

arXiv: https://arxiv.org/abs/2609.22486 · HTML: https://mlsanity.com/paper/2609.22486/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 32 of its 112 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 32)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Corrective Retrieval Augmented Generation](https://mlsanity.com/paper/2401.15884/) (2024)
- [ColPali: Efficient Document Retrieval with Vision Language Models](https://mlsanity.com/paper/2407.01449/) (2024)
- [On the Theoretical Limitations of Embedding-Based Retrieval](https://mlsanity.com/paper/2508.21038/) (2025)
- [R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning](https://mlsanity.com/paper/2505.17005/) (2025)
- [Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception](https://mlsanity.com/paper/2410.12788/) (2024)
- [Do Large Language Models Need a Content Delivery Network?](https://mlsanity.com/paper/2409.13761/) (2024)
- [ReaderLM-v2: Small Language Model for HTML to Markdown and JSON](https://mlsanity.com/paper/2503.01151/) (2025)
- [EdgeRAG: Online-Indexed RAG for Edge Devices](https://mlsanity.com/paper/2412.21023/) (2024)
- [Agentic-R: Learning to Retrieve for Agentic Search](https://mlsanity.com/paper/2601.11888/) (2026)
- [LEANN: A Low-Storage Vector Index](https://mlsanity.com/paper/2506.08276/) (2025)
- [WebLists: Extracting Structured Information From Complex Interactive Websites Using Executable LLM Agents](https://mlsanity.com/paper/2504.12682/) (2025)
- [Broken Gates: Re-evaluating Web Bot Defenses in the Age of LLM Agents](https://mlsanity.com/paper/2607.18659/) (2026)
- [An Index-based Approach for Efficient and Effective Web Content Extraction](https://mlsanity.com/paper/2512.06641/) (2025)
- [LURE-RAG: Lightweight Utility-driven Reranking for Efficient RAG](https://mlsanity.com/paper/2601.19535/) (2026)
- [PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.28344/) (2026)
- [Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs](https://mlsanity.com/paper/2607.04281/) (2026)
- [Spruce: Scalable Private Outsourced Retrieval Using Compact Embeddings](https://mlsanity.com/paper/2609.03376/) (2026)
- [WCXB: A Multi-Type Web Content Extraction Benchmark](https://mlsanity.com/paper/2605.21097/) (2026)
- [Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents](https://mlsanity.com/paper/2606.20910/) (2026)
- [Efficient and Effective In-place Graph-based Vector Index Updates](https://mlsanity.com/paper/2607.15576/) (2026)
- [Aker: Density-Aware Approximate Caching for Vector Search (Extended Version)](https://mlsanity.com/paper/2609.03712/) (2026)

## Built on by (most cited first, 2 of 2)

- [Backdoor in the Loop: Compromising Agentic Search via Malicious Retrievers](https://mlsanity.com/paper/2609.37468/) (2026)
- [Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG](https://mlsanity.com/paper/2609.37469/) (2026)

## Abstract (arXiv)

Large language models (LLMs) increasingly rely on external sources when answering questions that require proprietary information or up-to-date live web content, through both traditional single-shot retrieval-augmented generation (RAG) and multi-turn agentic RAG. Yet today's web infrastructure is still built for human clients. Given a query, current search services return a list of URLs and snippets ranked for generic relevance; content delivery networks (CDNs) cache URL-addressed objects (texts, images, videos, etc.) without knowing which passage an agent needs. LLMs, in contrast, consume short, semantically coherent passages, hereafter "chunks", selected for downstream task utility rather than similarity alone, and may retrieve statefully across reasoning turns. Uncoordinated agents also repeat search, data acquisition, and semantic processing, duplicating work that could be shared. We argue that semantic chunk retrieval should become a first-class network-delivery abstraction. We propose Semantics Delivery Network (SemDN): an origin-authorized, hierarchical edge substrate that indexes, searches, and smart-caches web content at chunk granularity. SemDN serves agents on behalf of participating websites, amortizes data acquisition and processing across agents, and supports tenant-specific retrieval policies. Because, unlike URL caching, semantic retrieval provides no explicit miss signal, SemDN must estimate when its enrolled corpus may be incomplete or stale and trigger scoped discovery or refresh. It raises open questions about shareable retrieval state, hierarchical caching, coverage risk, and deployment. Our preliminary probes reveal a large gap between page content processed and chunks consumed, substantial task-local reuse, and higher answer quality per context token from chunk delivery.