# fabric-lib: RDMA Point-to-Point Communication for LLM Systems

arXiv: https://arxiv.org/abs/2510.27656 · HTML: https://mlsanity.com/paper/2510.27656/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- First posted on arXiv 31 Oct 2025.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Built on by (most cited first, 8 of 8)

- [UCCL-EP: Portable Expert-Parallel Communication](https://mlsanity.com/paper/2512.19849/) (2025)
- [Role-Based Fault Tolerance System for LLM RL Post-Training](https://mlsanity.com/paper/2512.22492/) (2025)
- [Efficient Remote KV Cache Reuse with GPU-native Video Codec](https://mlsanity.com/paper/2602.09725/) (2026)
- [Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics](https://mlsanity.com/paper/2606.01502/) (2026)
- [Nereus: Adaptive Parallelism for LLM Post-Training](https://mlsanity.com/paper/2609.34645/) (2026)
- [The DMA Streaming Framework: Kernel-Level Buffer Orchestration for High-Performance AI Data Paths](https://mlsanity.com/paper/2603.10030/) (2026)
- [NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding](https://mlsanity.com/paper/2605.21100/) (2026)
- [WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learning](https://mlsanity.com/paper/2609.25442/) (2026)

## Abstract (arXiv)

Emerging Large Language Model (LLM) system patterns, such as disaggregated inference, Mixture-of-Experts (MoE) routing, and asynchronous reinforcement fine-tuning, require flexible point-to-point communication beyond simple collectives. Existing implementations are locked to specific Network Interface Controllers (NICs), hindering integration into inference engines and portability across hardware providers. We present fabric-lib, which bridges the functionality of common NICs to expose a uniform interface. fabric-lib exposes one-sided WriteImm operations with a ImmCounter primitive for completion notification, without ordering assumptions of network transport, transparently managing multiple NICs per GPU. We demonstrate peak throughput of 400 Gbps on both NVIDIA ConnectX-7 and AWS Elastic Fabric Adapter (EFA). We showcase fabric-lib through three production systems: (1) KvCache transfer for disaggregated inference with dynamic scaling, (2) RL weight updates achieving 1.3 seconds for trillion-parameter models, and (3) MoE dispatch/combine implementation exceeding DeepEP decode latency on ConnectX-7, with the first viable latencies on EFA. We demonstrate that our portable point-to-point communication complements collectives while avoiding lock-in. fabric-lib is open-sourced at https://github.com/perplexityai/pplx-garden/