# Towards Engineering Multi-Agent LLMs: A Protocol-Driven Approach

arXiv: https://arxiv.org/abs/2510.12120 · HTML: https://mlsanity.com/paper/2510.12120/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 16 of 16)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution](https://mlsanity.com/paper/2403.17927/) (2024)
- [A Survey of AI Agent Protocols](https://mlsanity.com/paper/2504.16736/) (2025)
- [Exploring and Unleashing the Power of Large Language Models in Automated Code Translation](https://mlsanity.com/paper/2404.14646/) (2024)
- [Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives](https://mlsanity.com/paper/2310.01152/) (2023)
- [CodeS: Natural Language to Code Repository via Multi-Layer Sketch](https://mlsanity.com/paper/2403.16443/) (2024)
- [MARE: Multi-Agents Collaboration Framework for Requirements Engineering](https://mlsanity.com/paper/2405.03256/) (2024)
- [A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement](https://mlsanity.com/paper/2409.05001/) (2024)
- [Multi-role Consensus through LLMs Discussions for Vulnerability Detection](https://mlsanity.com/paper/2403.14274/) (2024)
- [Elicitron: An LLM Agent-Based Simulation Framework for Design Requirements Elicitation](https://mlsanity.com/paper/2404.16045/) (2024)
- [INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair](https://mlsanity.com/paper/2311.09868/) (2023)
- [Exploring the Improvement of Evolutionary Computation via Large Language Models](https://mlsanity.com/paper/2405.02876/) (2024)
- [Hybrid Privacy Policy-Code Consistency Check using Knowledge Graphs and LLMs](https://mlsanity.com/paper/2505.11502/) (2025)

## Built on by (most cited first, 4 of 4)

- [One-Eval: An Agentic System for Automated and Traceable LLM Evaluation](https://mlsanity.com/paper/2603.09821/) (2026)
- [ATM: CID-Brokered Pre-Write Admission for Multi-Agent Code Co-Synthesis](https://mlsanity.com/paper/2607.00041/) (2026)
- [Contract-Coding: Towards Repo-Level Generation via Structured Symbolic Paradigm](https://mlsanity.com/paper/2604.13100/) (2026)
- [Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault](https://mlsanity.com/paper/2609.32802/) (2026)

## Abstract (arXiv)

The increasing demand for software development has driven interest in automating software engineering (SE) tasks using Large Language Models (LLMs). Recent efforts extend LLMs into multi-agent systems (MAS) that emulate collaborative development workflows, but these systems often fail due to three core deficiencies: under-specification, coordination misalignment, and inappropriate verification, arising from the absence of foundational SE structuring principles. This paper introduces Software Engineering Multi-Agent Protocol (SEMAP), a protocol-layer methodology that instantiates three core SE design principles for multi-agent LLMs: (1) explicit behavioral contract modeling, (2) structured messaging, and (3) lifecycle-guided execution with verification, and is implemented atop Google's Agent-to-Agent (A2A) infrastructure. Empirical evaluation using the Multi-Agent System Failure Taxonomy (MAST) framework demonstrates that SEMAP effectively reduces failures across different SE tasks. In code development, it achieves up to a 69.6% reduction in total failures for function-level development and 56.7% for deployment-level development. For vulnerability detection, SEMAP reduces failure counts by up to 47.4% on Python tasks and 28.2% on C/C++ tasks.