# Large Language Model Based Multi-Agent System Augmented Complex Event Processing Pipeline for Internet of Multimedia Things

arXiv: https://arxiv.org/abs/2501.00906 · HTML: https://mlsanity.com/paper/2501.00906/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 1 Jan 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions](https://mlsanity.com/paper/2306.02224/) (2023)
- [SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2309.10062/) (2023)
- [Large Language Models for Robotics: Opportunities, Challenges, and Perspectives](https://mlsanity.com/paper/2401.04334/) (2024)
- [Towards an Understanding of Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2308.11396/) (2023)
- [AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System](https://mlsanity.com/paper/2402.15538/) (2024)
- [LLM Harmony: Multi-Agent Communication for Problem Solving](https://mlsanity.com/paper/2401.01312/) (2024)
- [AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology](https://mlsanity.com/paper/2306.10095/) (2023)
- [ChatGPT: Applications, Opportunities, and Threats](https://mlsanity.com/paper/2304.09103/) (2023)
- [AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaboration](https://mlsanity.com/paper/2404.11943/) (2024)
- [LLM-based Robot Task Planning with Exceptional Handling for General Purpose Service Robots](https://mlsanity.com/paper/2405.15646/) (2024)
- [Smurfs: Multi-Agent System using Context-Efficient DFSDT for Tool Planning](https://mlsanity.com/paper/2405.05955/) (2024)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Large Language Models meet Network Slicing Management and Orchestration](https://mlsanity.com/paper/2403.13721/) (2024)
- [Towards Message Brokers for Generative AI: Survey, Challenges, and Opportunities](https://mlsanity.com/paper/2312.14647/) (2023)
- [How Can AI be Distributed in the Computing Continuum? Introducing the Neural Pub/Sub Paradigm](https://mlsanity.com/paper/2309.02058/) (2023)
- [A Manifesto for Future Generation Cloud Computing: Research Directions for the Next Decade](https://mlsanity.com/paper/1711.09123/) (2017)

## Built on by (most cited first, 8 of 8)

- [AgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System Need](https://mlsanity.com/paper/2506.15451/) (2025)
- [Advanced Architectures Integrated with Agentic AI for Next-Generation Wireless Networks](https://mlsanity.com/paper/2502.01089/) (2025)
- [Position: Towards a Responsible LLM-empowered Multi-Agent Systems](https://mlsanity.com/paper/2502.01714/) (2025)
- [Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignment](https://mlsanity.com/paper/2510.14008/) (2025)
- [From Virtual Agents to Robot Teams: A Multi-Robot Framework Evaluation in High-Stakes Healthcare Context](https://mlsanity.com/paper/2506.03546/) (2025)
- [Multi-dimensional Data Analysis and Applications Basing on LLM Agents and Knowledge Graph Interactions](https://mlsanity.com/paper/2510.15258/) (2025)
- [GoalfyMax: A Protocol-Driven Multi-Agent System for Intelligent Experience Entities](https://mlsanity.com/paper/2507.09497/) (2025)
- [IMAGINE: Integrating Multi-Agent System into One Model for Complex Reasoning and Planning](https://mlsanity.com/paper/2510.14406/) (2025)

## Abstract (arXiv)

This paper presents the development and evaluation of a Large Language Model (LLM), also known as foundation models, based multi-agent system framework for complex event processing (CEP) with a focus on video query processing use cases. The primary goal is to create a proof-of-concept (POC) that integrates state-of-the-art LLM orchestration frameworks with publish/subscribe (pub/sub) tools to address the integration of LLMs with current CEP systems. Utilizing the Autogen framework in conjunction with Kafka message brokers, the system demonstrates an autonomous CEP pipeline capable of handling complex workflows. Extensive experiments evaluate the system's performance across varying configurations, complexities, and video resolutions, revealing the trade-offs between functionality and latency. The results show that while higher agent count and video complexities increase latency, the system maintains high consistency in narrative coherence. This research builds upon and contributes to, existing novel approaches to distributed AI systems, offering detailed insights into integrating such systems into existing infrastructures.