mlsanityData to 14 Oct 2026

LLM-Based Code Documentation Generation and Multi-Judge Evaluation

arXiv 2606.09852 ↗First posted 11 May 2026cs.HC cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

High-quality source code documentation is vital yet often neglected, especially in critical domains like healthcare where reliability and maintainability are essential. We presented an AI powered framework that automates documentation generation from code and repositories using eight state of the art Large Language Models (LLMs), including GPT, Gemini, Qwen, and LLaMA variants. Built on the PocketFlow orchestration framework, the system applies modular pipelines and advanced prompt engineering to produce structured, context aware documentation. To ensure quality and guide model selection, we introduced a MultiLLMasJudges evaluation framework, where four independent LLMs assess outputs across nine criteria, such as Completeness, Clarity, and Faithfulness. Experiments conducted on an open-source medical physics library, demonstrated showed a 42% performance gap between top and bottom models. By combining diverse model outputs, optimized prompting, and rigorous evaluation, our approach enhances documentation quality and reduces manual effort, especially in safety critical healthcare software.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 15 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

7 of 15 references.

References · 7

  1. 2024A Survey on Evaluating Large Language Models in Code Generation Taskscited by 44
  2. 2023Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMscited by 24
  3. 2025Large Language Models (LLMs) for Source Code Analysis: applications, models and datasetscited by 16
  4. 2024Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approachcited by 10
  5. 2024A review of faithfulness metrics for hallucination assessment in Large Language Modelscited by 9
  6. 2022Detecting Outdated Code Element References in Software Repository Documentationcited by 5
  7. 2025Flow State: Humans Enabling AI Systems to Program Themselvescited by 3

Built on by

Not cited yet.