Whitespaces Don't Lie: Feature-Driven and Embedding-Based Approaches for Detecting Machine-Generated Code
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) have made it remarkably easy to synthesize plausible source code from natural language prompts. While this accelerates software development and supports learning, it also raises new risks for academic integrity, authorship attribution, and responsible AI use. This paper investigates the problem of distinguishing human-written from machine-generated code by comparing two complementary approaches: feature-based detectors built from lightweight, interpretable stylometric and structural properties of code, and embedding-based detectors leveraging pretrained code encoders. Using a recent large-scale benchmark dataset of 600k human-written and AI-generated code samples, we find that feature-based models achieve strong performance (ROC-AUC 0.995, PR-AUC 0.995, F1 0.971), while embedding-based models with CodeBERT embeddings are also very competitive (ROC-AUC 0.994, PR-AUC 0.994, F1 0.965). Analysis shows that features tied to indentation and whitespace provide particularly discriminative cues, whereas embeddings capture deeper semantic patterns and yield slightly higher precision. These findings underscore the trade-offs between interpretability and generalization, offering practical guidance for deploying robust code-origin detection in academic and industrial contexts.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 27 Jan 2026.
Builds on
11 of 20 references.
References · 11
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
- 2019Release Strategies and the Social Impacts of Language Modelscited by 505
- 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
- 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
- 2024Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmerscited by 27
- 2021Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programmingcited by 24
- 2023Zero-Shot Detection of Machine-Generated Codescited by 24
- 2023Practices and Challenges of Using GitHub Copilot: An Empirical Studycited by 13
- 2025CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settingscited by 11
- 2025textttDroid: A Resource Suite for AI-Generated Code Detectioncited by 9
Built on by
Not cited yet.