AICD Bench: A Challenging Benchmark for AI-Generated Code Detection
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly capable of generating functional source code, raising concerns about authorship, accountability, and security. While detecting AI-generated code is critical, existing datasets and benchmarks are narrow, typically limited to binary human-machine classification under in-distribution settings. To bridge this gap, we introduce AICD Bench, the most comprehensive benchmark for AI-generated code detection. It spans 2M examples, 77 models across 11 families, and 9 programming languages, including recent reasoning models. Beyond scale, AICD Bench introduces three realistic detection tasks: (i)~Robust Binary Classification under distribution shifts in language and domain, (ii)~Model Family Attribution, grouping generators by architectural lineage, and (iii)~Fine-Grained Human-Machine Classification across human, machine, hybrid, and adversarial code. Extensive evaluation on neural and classical detectors shows that performance remains far below practical usability, particularly under distribution shift and for hybrid or adversarial code. We release AICD Bench as a unified, challenging evaluation suite to drive the next generation of robust approaches for AI-generated code detection. The data and the code are available at https://huggingface.co/AICD-bench.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 63 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
36 of 63 references.
References · 36
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2024Yi: Open Foundation Models by 01.AIcited by 706
- 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
- 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
- 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
24 more
- 2022The Stack: 3 TB of permissively licensed source codecited by 354
- 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 278
- 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
- 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 157
- 2024CodeGemma: Open Code Models Based on Gemmacited by 145
- 2023MAGE: Machine-generated Text Detection in the Wildcited by 130
- 2020Unit Test Case Generation with Transformers and Focal Contextcited by 118
- 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligencecited by 116
- 2024Few-Shot Detection of Machine-Generated Text using Style Representationscited by 40
- 2023GPT-who: An Information Density-based Machine-Generated Text Detectorcited by 38
- 2025Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligencecited by 20
- 2025Rewriting Pre-Training Data Boosts LLM Performance in Math and Codecited by 17
- 2024Distinguishing LLM-generated from Human-written Code by Contrastive Learningcited by 16
- 2024Whodunit: Classifying Code as Human Authored or GPT-4 Generated -- A case study on CodeChef problemscited by 13
- 2022Generalization in Neural Networks: A Broad Surveycited by 11
- 2025CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settingscited by 11
- 2015The Singular Value Decomposition, Applications and Beyondcited by 10
- 2024Natural Language Outlines for Code: Literate Programming in the LLM Eracited by 9
- 2025textttDroid: A Resource Suite for AI-Generated Code Detectioncited by 9
- 2025CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMscited by 6
- 2024Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbationcited by 5
- 2025Detection of LLM-Paraphrased Code and Identification of the Responsible LLM Using Coding Style Featurescited by 4
- 2025ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Groundingcited by 4
- 2014A Model of the Commit Size Distribution of Open Sourcecited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2023ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
- 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
- 2025PACIFIC: a framework for generating benchmarks to check Precise Automatically Checked Instruction Following In Code
- 2025Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
- 2024Benchmarking Large Language Models with Integer Sequence Generation Tasks
- 2024AICoderEval: Improving AI Domain Code Generation of Large Language Models
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- 2026Beyond Problem Solving: UOJ-Bench for Evaluating Code Generation, Hacking, and Repair in Competitive Programming