# MARCO: Multi-Agent Real-time Chat Orchestration

arXiv: https://arxiv.org/abs/2410.21784 · HTML: https://mlsanity.com/paper/2410.21784/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [Agents: An Open-source Framework for Autonomous Language Agents](https://mlsanity.com/paper/2309.07870/) (2023)
- [Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks](https://mlsanity.com/paper/2305.14965/) (2023)
- [Reflective Linguistic Programming (RLP): A Stepping Stone in Socially-Aware AGI (SocialAGI)](https://mlsanity.com/paper/2305.12647/) (2023)

## Built on by (most cited first, 5 of 5)

- [Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms](https://mlsanity.com/paper/2507.06323/) (2025)
- [PARSE: LLM Driven Schema Optimization for Reliable Entity Extraction](https://mlsanity.com/paper/2510.08623/) (2025)
- [A Practical Approach for Building Production-Grade Conversational Agents with Workflow Graphs](https://mlsanity.com/paper/2505.23006/) (2025)
- [CineForge: Self-Improving Agents for Long-Horizon Video Generation](https://mlsanity.com/paper/2608.29621/) (2026)
- [A Scalable Trust Discovery Architecture for the Internet of Agents](https://mlsanity.com/paper/2609.20095/) (2026)

## Abstract (arXiv)

Large language model advancements have enabled the development of multi-agent frameworks to tackle complex, real-world problems such as to automate tasks that require interactions with diverse tools, reasoning, and human collaboration. We present MARCO, a Multi-Agent Real-time Chat Orchestration framework for automating tasks using LLMs. MARCO addresses key challenges in utilizing LLMs for complex, multi-step task execution. It incorporates robust guardrails to steer LLM behavior, validate outputs, and recover from errors that stem from inconsistent output formatting, function and parameter hallucination, and lack of domain knowledge. Through extensive experiments we demonstrate MARCO's superior performance with 94.48% and 92.74% accuracy on task execution for Digital Restaurant Service Platform conversations and Retail conversations datasets respectively along with 44.91% improved latency and 33.71% cost reduction. We also report effects of guardrails in performance gain along with comparisons of various LLM models, both open-source and proprietary. The modular and generic design of MARCO allows it to be adapted for automating tasks across domains and to execute complex usecases through multi-turn interactions.