# Human-Written vs. AI-Generated Code: A Large-Scale Study of Defects, Vulnerabilities, and Complexity

arXiv: https://arxiv.org/abs/2508.21634 · HTML: https://mlsanity.com/paper/2508.21634/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2025.

## Built on by (most cited first, 16 of 16)

- [SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks](https://mlsanity.com/paper/2603.24755/) (2026)
- [Measuring AI R&D Automation](https://mlsanity.com/paper/2603.03992/) (2026)
- [What to Cut? Predicting Unnecessary Methods in Agentic Code Generation](https://mlsanity.com/paper/2602.17091/) (2026)
- [Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests](https://mlsanity.com/paper/2601.17627/) (2026)
- [Let's Make Every Pull Request Meaningful: An Empirical Analysis of Developer and Agentic Pull Requests](https://mlsanity.com/paper/2601.18749/) (2026)
- [To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study](https://mlsanity.com/paper/2605.06464/) (2026)
- [PIE: Performance Interval Estimation for Free-Form Generation Tasks](https://mlsanity.com/paper/2509.07309/) (2025)
- [Mining Architectural Quality Under Agentic AI Adoption: A Causal Study of Java Repositories](https://mlsanity.com/paper/2606.13298/) (2026)
- [Towards an Agent Operating System - Lessons from Classical and Cloud OS](https://mlsanity.com/paper/2607.25076/) (2026)
- [Where Did the Variability Go? From Vibe Coding to Product Lines by Regeneration](https://mlsanity.com/paper/2606.19042/) (2026)
- [Enhancing SLMs for Sustainable Code Optimization in Radio-Astronomy](https://mlsanity.com/paper/2607.21677/) (2026)
- [Generation of Web Apps with Agentic IDEs: An Empirical Assessment](https://mlsanity.com/paper/2608.20903/) (2026)
- [AI Writes Code, Humans Pay the Debt. An Empirical Study on the Sustainability and Evolution of Agent-Generated Code](https://mlsanity.com/paper/2609.04208/) (2026)
- [What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code](https://mlsanity.com/paper/2609.12708/) (2026)
- [Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase](https://mlsanity.com/paper/2609.29744/) (2026)
- [CRJudgeBench: Can AI Detect Plausible but Invalid Code Reviews?](https://mlsanity.com/paper/2609.37216/) (2026)

## Abstract (arXiv)

As AI code assistants become increasingly integrated into software development workflows, understanding how their code compares to human-written programs is critical for ensuring reliability, maintainability, and security. In this paper, we present a large-scale comparison of code authored by human developers and three state-of-the-art LLMs, i.e., ChatGPT, DeepSeek-Coder, and Qwen-Coder, on multiple dimensions of software quality: code defects, security vulnerabilities, and structural complexity. Our evaluation spans over 500k code samples in two widely used languages, Python and Java, classifying defects via Orthogonal Defect Classification and security vulnerabilities using the Common Weakness Enumeration. We find that AI-generated code is generally simpler and more repetitive, yet more prone to unused constructs and hardcoded debugging, while human-written code exhibits greater structural complexity and a higher concentration of maintainability issues. Notably, AI-generated code also contains more high-risk security vulnerabilities. These findings highlight the distinct defect profiles of AI- and human-authored code and underscore the need for specialized quality assurance practices in AI-assisted programming.