# The Illusion of Agentic Complexity in README.md Generation: Evaluating Single-Agent vs. Multi-Agent RAG Systems

arXiv: https://arxiv.org/abs/2606.30524 · HTML: https://mlsanity.com/paper/2606.30524/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG for code, one of 15 directions in Retrieval-augmented generation.
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- Retrieval-augmented generation / [RAG for code](https://mlsanity.com/rag/rag-for-code/)
- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2406.07791/) (2024)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution](https://mlsanity.com/paper/2403.17927/) (2024)
- [Repository-Level Prompt Generation for Large Language Models of Code](https://mlsanity.com/paper/2206.12839/) (2022)
- [RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation](https://mlsanity.com/paper/2402.16667/) (2024)
- [Comparing Code Explanations Created by Students and Large Language Models](https://mlsanity.com/paper/2304.03938/) (2023)
- [Automatic Code Documentation Generation Using GPT-3](https://mlsanity.com/paper/2209.02235/) (2022)
- [A Survey on Data Contamination for Large Language Models](https://mlsanity.com/paper/2502.14425/) (2025)
- [Single-agent or Multi-agent Systems? Why Not Both?](https://mlsanity.com/paper/2505.18286/) (2025)
- [Understanding the Factors that Impact the Popularity of GitHub Repositories](https://mlsanity.com/paper/1606.04984/) (2016)
- [A Comparative Analysis of Large Language Models for Code Documentation Generation](https://mlsanity.com/paper/2312.10349/) (2023)
- [DocAgent: A Multi-Agent System for Automated Code Documentation Generation](https://mlsanity.com/paper/2504.08725/) (2025)
- [Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approach](https://mlsanity.com/paper/2412.00726/) (2024)
- [Teamwork makes the dream work: LLMs-Based Agents for GitHub README.MD Summarization](https://mlsanity.com/paper/2503.10876/) (2025)
- [CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases](https://mlsanity.com/paper/2510.24428/) (2025)
- [Context-aware Code Summary Generation](https://mlsanity.com/paper/2408.09006/) (2024)
- [RepoSummary: Feature-Oriented Summarization and Documentation Generation for Code Repositories](https://mlsanity.com/paper/2510.11039/) (2025)
- [LARCH: Large Language Model-based Automatic Readme Creation with Heuristics](https://mlsanity.com/paper/2308.03099/) (2023)
- [Automated and Context-Aware Code Documentation Leveraging Advanced LLMs](https://mlsanity.com/paper/2509.14273/) (2025)
- [Does My README File Need To Be Updated? Exploring LLM-Based README Maintenance](https://mlsanity.com/paper/2603.00489/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly utilized to automate several software engineering tasks, including code completion, code summarization, testing, and the generation of repository-level documentation. While Multi-Agent Systems (MAS) are often adopted to support such tasks under the premise that task decomposition improves performance, the impact of architectural complexity on practical efficiency remains under-examined. This study empirically evaluates Retrieval-Augmented Generation (RAG) dependent architectures for the generation of README files for GitHub repositories. In this work, we conducted a systematic comparison between a Single-Agent pipeline, a specialized MAS, and a developer-guided planning (DevPlan) variant, benchmarked against LARCH -- a state-of-the-art baseline -- and the original ground truth. Results indicate a critical architectural trade-off: the Single-Agent pipeline achieves lexical quality comparable to MAS while reducing token consumption by 86% and operating at twice the speed. In contrast, manual taxonomy analysis demonstrates that MAS achieves high structural consistency (98%), resolving formatting issues observed in single-agent approaches. Autonomous planning is identified as the primary pipeline bottleneck; incorporating lightweight developer-guided plans produces the highest overall documentation quality, surpassing all the analyzed configurations.