The Cost of a Hop: Benchmarking NLIP and A2A
Not on the map. Why?
Abstract (arXiv)
Autonomous agents built on Large Language Models (LLMs) need standardized protocols to interoperate across systems. Several now exist (A2A, MCP, ACP, ANP, NLIP), but the Natural Language Interaction Protocol (NLIP) has not appeared in any controlled performance study, and no work has measured where an agent protocol's latency is spent. We compare NLIP and the Agent-to-Agent (A2A) protocol empirically, decomposing latency into message creation, connection, and send phases across three independent hardware environments. For lightweight coordination, NLIP is 8.4-9.6x faster than the baseline A2A SDK implementation on two environments and about 4x on a third; the direction of the advantage is consistent, its magnitude depends on the hardware. The advantage is stage-specific: for the end-to-end pipeline, where LLM inference dominates, the protocols are near parity. The difference comes almost entirely from connection setup. To test A2A at its best, we also ran A2A SDK with connection caching enabled; caching narrows its gap with NLIP by a hardware-dependent amount, from 2.75x on one machine to near-parity on faster hardware, where at scale a cache-optimized A2A-SDK matches NLIP. We report these as measured conditions without a single causal account of the residual send-phase cost. Against the more optimized Python-A2A, NLIP leads by about 4x on the same stage. We close with a protocol-selection guide keyed to workload characteristics.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 2 Oct 2026.
Builds on
21 of 33 references.
References · 21
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 265
9 more
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 178
- 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 141
- 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 109
- 2025A Survey of AI Agent Protocolscited by 81
- 2024LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Acceleratorscited by 32
- 2025On Evaluating Performance of LLM Inference Serving Systemscited by 10
- 2025ProtocolBench: Which LLM MultiAgent Protocol to Choose?cited by 6
- 2022Eficiency of REST and gRPC realizing communication tasks in microservice-based ecosystemscited by 2
- 2026The Natural Language Interaction Protocol and Standard for AI Agentscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
- 2026Empirical Comparison of Agent Communication Protocols for Task Orchestration
- 2026UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
- 2026Beyond Message Passing: A Semantic View of Agent Communication Protocols
- 2026A Comparative Study of MCP and A2A for Inter-Agent Coordination in LLM-Based Systems
- 2026BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents
- 2026A Technical Taxonomy of LLM Agent Communication Protocols
- 2026Are We Ready For An Agent-Native Memory System?