mlsanityData to 7 Oct 2026

The Illusion of Agentic Complexity in README.md Generation: Evaluating Single-Agent vs. Multi-Agent RAG Systems

arXiv 2606.30524 ↗First posted 29 Jun 2026cs.SEcited by 0

On the map: Retrieval-augmented generation → RAG for code

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) are increasingly utilized to automate several software engineering tasks, including code completion, code summarization, testing, and the generation of repository-level documentation. While Multi-Agent Systems (MAS) are often adopted to support such tasks under the premise that task decomposition improves performance, the impact of architectural complexity on practical efficiency remains under-examined. This study empirically evaluates Retrieval-Augmented Generation (RAG) dependent architectures for the generation of README files for GitHub repositories. In this work, we conducted a systematic comparison between a Single-Agent pipeline, a specialized MAS, and a developer-guided planning (DevPlan) variant, benchmarked against LARCH -- a state-of-the-art baseline -- and the original ground truth. Results indicate a critical architectural trade-off: the Single-Agent pipeline achieves lexical quality comparable to MAS while reducing token consumption by 86% and operating at twice the speed. In contrast, manual taxonomy analysis demonstrates that MAS achieves high structural consistency (98%), resolving formatting issues observed in single-agent approaches. Autonomous planning is identified as the primary pipeline bottleneck; incorporating lightweight developer-guided plans produces the highest overall documentation quality, surpassing all the analyzed configurations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 36 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

27 of 36 references.

References · 27

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  6. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  7. 2024A Survey on Large Language Models for Code Generationcited by 646
  8. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  9. 2023Using an LLM to Help With Code Understandingcited by 222
  10. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
  11. 2022Repository-Level Prompt Generation for Large Language Models of Codecited by 106
  12. 2024RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generationcited by 69
15 more
  1. 2023Comparing Code Explanations Created by Students and Large Language Modelscited by 63
  2. 2022Automatic Code Documentation Generation Using GPT-3cited by 37
  3. 2025A Survey on Data Contamination for Large Language Modelscited by 33
  4. 2025Single-agent or Multi-agent Systems? Why Not Both?cited by 28
  5. 2016Understanding the Factors that Impact the Popularity of GitHub Repositoriescited by 27
  6. 2023A Comparative Analysis of Large Language Models for Code Documentation Generationcited by 24
  7. 2025DocAgent: A Multi-Agent System for Automated Code Documentation Generationcited by 16
  8. 2024Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approachcited by 10
  9. 2025Teamwork makes the dream work: LLMs-Based Agents for GitHub README.MD Summarizationcited by 4
  10. 2025CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebasescited by 4
  11. 2024Context-aware Code Summary Generationcited by 3
  12. 2025RepoSummary: Feature-Oriented Summarization and Documentation Generation for Code Repositoriescited by 3
  13. 2023LARCH: Large Language Model-based Automatic Readme Creation with Heuristicscited by 2
  14. 2025Automated and Context-Aware Code Documentation Leveraging Advanced LLMscited by 2
  15. 2026Does My README File Need To Be Updated? Exploring LLM-Based README Maintenancecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
  2. 2025LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
  3. 2026CodeTeam: An LLM-Powered Multi-Agent Framework for Repository-Level Code Generation
  4. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  5. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges
  6. 2026Is Agentic RAG worth it? An experimental comparison of RAG approaches
  7. 2026Developing LLM-based Multi-Agent Systems in Software Engineering: A Mixed-Method Experience Report
  8. 2024LLM Agents Improve Semantic Code Search