# AICD Bench: A Challenging Benchmark for AI-Generated Code Detection

arXiv: https://arxiv.org/abs/2602.02079 · HTML: https://mlsanity.com/paper/2602.02079/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 63 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 30 of 36)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks](https://mlsanity.com/paper/2105.12655/) (2021)
- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [CodeGemma: Open Code Models Based on Gemma](https://mlsanity.com/paper/2406.11409/) (2024)
- [MAGE: Machine-generated Text Detection in the Wild](https://mlsanity.com/paper/2305.13242/) (2023)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [Granite Code Models: A Family of Open Foundation Models for Code Intelligence](https://mlsanity.com/paper/2405.04324/) (2024)
- [GPT-who: An Information Density-based Machine-Generated Text Detector](https://mlsanity.com/paper/2310.06202/) (2023)
- [Few-Shot Detection of Machine-Generated Text using Style Representations](https://mlsanity.com/paper/2401.06712/) (2024)
- [Rewriting Pre-Training Data Boosts LLM Performance in Math and Code](https://mlsanity.com/paper/2505.02881/) (2025)
- [Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence](https://mlsanity.com/paper/2502.09927/) (2025)
- [Distinguishing LLM-generated from Human-written Code by Contrastive Learning](https://mlsanity.com/paper/2411.04704/) (2024)
- [Whodunit: Classifying Code as Human Authored or GPT-4 Generated -- A case study on CodeChef problems](https://mlsanity.com/paper/2403.04013/) (2024)
- [The Singular Value Decomposition, Applications and Beyond](https://mlsanity.com/paper/1510.08532/) (2015)
- [Generalization in Neural Networks: A Broad Survey](https://mlsanity.com/paper/2209.01610/) (2022)
- [CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settings](https://mlsanity.com/paper/2503.13733/) (2025)
- [Natural Language Outlines for Code: Literate Programming in the LLM Era](https://mlsanity.com/paper/2408.04820/) (2024)

## Built on by (most cited first, 2 of 2)

- [Dream at SemEval-2026 Task 13: SALSA for Single-Pass Machine-Generated Code Detection](https://mlsanity.com/paper/2606.25102/) (2026)
- [Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models](https://mlsanity.com/paper/2609.30048/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly capable of generating functional source code, raising concerns about authorship, accountability, and security. While detecting AI-generated code is critical, existing datasets and benchmarks are narrow, typically limited to binary human-machine classification under in-distribution settings. To bridge this gap, we introduce AICD Bench, the most comprehensive benchmark for AI-generated code detection. It spans 2M examples, 77 models across 11 families, and 9 programming languages, including recent reasoning models. Beyond scale, AICD Bench introduces three realistic detection tasks: (i)~Robust Binary Classification under distribution shifts in language and domain, (ii)~Model Family Attribution, grouping generators by architectural lineage, and (iii)~Fine-Grained Human-Machine Classification across human, machine, hybrid, and adversarial code. Extensive evaluation on neural and classical detectors shows that performance remains far below practical usability, particularly under distribution shift and for hybrid or adversarial code. We release AICD Bench as a unified, challenging evaluation suite to drive the next generation of robust approaches for AI-generated code detection. The data and the code are available at https://huggingface.co/AICD-bench.