# Understanding the Impact of AI Code Assistants on Security API Usage: An Empirical Study

arXiv: https://arxiv.org/abs/2607.11348 · HTML: https://mlsanity.com/paper/2607.11348/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 23 of 23)

- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Do Users Write More Insecure Code with AI Assistants?](https://mlsanity.com/paper/2211.03622/) (2022)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [Productivity Assessment of Neural Code Completion](https://mlsanity.com/paper/2205.06537/) (2022)
- [Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants](https://mlsanity.com/paper/2208.09727/) (2022)
- [How Secure is Code Generated by ChatGPT?](https://mlsanity.com/paper/2304.09655/) (2023)
- [Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?](https://mlsanity.com/paper/2204.04741/) (2022)
- [Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code](https://mlsanity.com/paper/2003.07914/) (2020)
- [No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT](https://mlsanity.com/paper/2308.04838/) (2023)
- [Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study](https://mlsanity.com/paper/2310.02059/) (2023)
- [Constrained Decoding for Secure Code Generation](https://mlsanity.com/paper/2405.00218/) (2024)
- [Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns](https://mlsanity.com/paper/2405.06371/) (2024)
- [A User-centered Security Evaluation of Copilot](https://mlsanity.com/paper/2308.06587/) (2023)
- [Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models](https://mlsanity.com/paper/2502.06039/) (2025)
- [An Investigation into Misuse of Java Security APIs by Large Language Models](https://mlsanity.com/paper/2404.03823/) (2024)
- [CASTLE: Benchmarking Dataset for Static Code Analyzers and LLMs towards CWE Detection](https://mlsanity.com/paper/2503.09433/) (2025)
- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [CryptoGuard: High Precision Detection of Cryptographic Vulnerabilities in Massive-sized Java Projects](https://mlsanity.com/paper/1806.06881/) (2018)
- [Systematic Literature Review on Application of Learning-based Approaches in Continuous Integration](https://mlsanity.com/paper/2406.19765/) (2024)
- [SoK: Machine Learning for Continuous Integration](https://mlsanity.com/paper/2304.02829/) (2023)
- [Detecting Misuse of Security APIs: A Systematic Review](https://mlsanity.com/paper/2306.08869/) (2023)
- [A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality](https://mlsanity.com/paper/2408.00435/) (2024)
- [Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2604.19024/) (2026)

## Abstract (arXiv)

AI code assistants are transforming software development, but their implications for software security remain a major concern, particularly in the context of security APIs. These APIs are critical for safeguarding software systems, yet their complexity often leads to incorrect use and serious vulnerabilities. Developing an evidence-based understanding of how AI assistants influence developers' use of these APIs is therefore essential for informing effective mitigation strategies. While a few user studies have examined the broader impact of AI assistants on software vulnerabilities, the use of security APIs remains unexplored from a developer-centered perspective. This study addresses this gap by presenting the first empirical investigation into how AI code assistants affect professional developers' use of security APIs. We conducted a study with 44 developers who completed security API programming tasks with and without GitHub Copilot assistance. Our findings show that, while Copilot improves functional correctness and marginally reduces certain insecure patterns, it does not significantly improve secure API usage. We also found that developers rarely raised security concerns when engaging with Copilot, and many did not recognize that their final implementations remained insecure. Finally, we offer recommendations for enhancing security awareness among developers and propose future research directions to support safer AI-assisted software development.