mlsanityData to 14 Oct 2026

Tiara: A Programmable Line-Rate ISA for Remote Memory Access

arXiv 2606.13708 ↗First posted 10 Jun 2026cs.AR cs.NIcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

RDMA one-sided verbs are the natural primitive for memory disaggregation, but they require the client to supply the exact remote address. The 1-RTT performance breaks down when the target address depends on data that must first be read from remote memory, a pattern we call the Indirection Wall. Indirection is pervasive: graph traversals follow pointers hop by hop, address translation walks multi-level page tables, distributed coordination requires conditional multi-host logic, and disaggregated LLM inference must resolve paged KV caches through block-table lookups. Each level of indirection costs one sequentially dependent network round-trip, yet offloading to existing RDMA NICs either consumes remote CPU cycles or has limited throughput. We present Tiara, a compact, statically verifiable instruction set that executes on the memory-side NIC. Tiara operators are pre-registered programs, analogous to eBPF programs in the kernel, that resolve indirection locally, collapsing multi-RTT dependent chains into a single round-trip. On an FPGA-based prototype, Tiara reduces 10-hop graph-traversal latency by 2.85x over one-sided RDMA while sustaining 3.4x higher throughput, cuts page-table walk latency by 62%, reduces uncontended distributed-lock latency by 2.9x, achieves 2.8x throughput for disaggregated PagedAttention at 8 KB blocks, and 1.88x MoE expert-gather latency at 32 experts.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 6 of its 49 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

6 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 5

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  3. 2013Programming Protocol-Independent Packet Processorscited by 21
  4. 2019Storm: a fast transactional dataplane for remote data structurescited by 1
  5. 2024Disaggregating Embedding Recommendation Systems with FlexEMRcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC
  2. 2026Rethinking Unified Memory for NPU-PIM Systems: Dual-View Memory for Dynamic Inference of LLM ↗
  3. 2026Budgeting Bytes: A Windowed Storage Roofline and Dual-Budget Architecture Ablations for Storage-Bound LLM Decoding
  4. 2025FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
  5. 2026NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference
  6. 2026From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving
  7. 2026HyperOffload: Graph-Driven Hierarchical Memory Management for Large Language Models on SuperNode Architectures
  8. 2025TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale