SoK: DARPA's AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned
Not on the map. Why?
Abstract (arXiv)
DARPA's AI Cyber Challenge (AIxCC, 2023--2025) is the largest competition to date for building fully autonomous cyber reasoning systems (CRSs) that leverage recent advances in AI -- particularly large language models (LLMs) -- to discover and remediate vulnerabilities in real-world open-source software. This paper presents the first systematic analysis of AIxCC. Drawing on design documents, source code, execution traces, and discussions with organizers and competing teams, we examine the competition's structure and key design decisions, characterize the architectural approaches of finalist CRSs, and analyze competition results beyond the final scoreboard. Our analysis reveals the factors that truly drove CRS performance, identifies genuine technical advances achieved by teams, and exposes limitations that remain open for future research. We conclude with lessons for organizing future competitions and broader insights toward deploying autonomous CRSs in practice.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 10 of its 69 references that are in the map. First posted on arXiv 7 Feb 2026.
Builds on
10 of 69 references.
References · 10
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2025ATLANTIS: AI-driven Threat Localization, Analysis, and Triage Intelligence Systemcited by 9
- 2023autofz: Automated Fuzzer Composition at Runtimecited by 3
- 2025BandFuzz: An ML-powered Collaborative Fuzzing Frameworkcited by 2
- 2026PatchIsland: Orchestration of LLM Agents for Continuous Vulnerability Repaircited by 2
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generationcited by 3
- 2026Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalationcited by 1
- 2026Revelio: Cost-Efficient Agentic Memory Safety Vulnerability Detection For Repository-Scale Codebasescited by 1
- 2026Code-Augur: Agentic Vulnerability Detection via Specification Inferencecited by 0
- 2026Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilitiescited by 0
- 2026PatchBench: Evaluating AI Agents for Vulnerability Patchingcited by 0
- 2026The History Is the Detector: Executing CVE Patch History, End-to-Endcited by 0
- 2026SelfOp: An Optimization Algorithm for Self-Improving Security Agentscited by 0
- 2026MobileCybench: Evaluating Agent Vulnerability Discovery via Executable Probescited by 0
- 2026MobileCybench: Evaluating Agent Vulnerability Discovery via Executable Probes21 Sep 2026
- 2026SelfOp: An Optimization Algorithm for Self-Improving Security Agents19 Sep 2026
- 2026The History Is the Detector: Executing CVE Patch History, End-to-End4 Sep 2026
- 2026PatchBench: Evaluating AI Agents for Vulnerability Patching3 Sep 2026
- 2026Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities25 Jun 2026
- 2026Revelio: Cost-Efficient Agentic Memory Safety Vulnerability Detection For Repository-Scale Codebases20 Jun 2026
- 2026Code-Augur: Agentic Vulnerability Detection via Specification Inference17 Jun 2026
- 2026Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation18 Mar 2026
- 2026Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generation14 Feb 2026