# A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality

arXiv: https://arxiv.org/abs/2408.00435 · HTML: https://mlsanity.com/paper/2408.00435/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design](https://mlsanity.com/paper/2303.07839/) (2023)
- [Data Quality for Software Vulnerability Datasets](https://mlsanity.com/paper/2301.05456/) (2023)
- [Prompt-Enhanced Software Vulnerability Detection Using ChatGPT](https://mlsanity.com/paper/2308.12697/) (2023)
- [No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT](https://mlsanity.com/paper/2308.04838/) (2023)
- [ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?](https://mlsanity.com/paper/2310.09810/) (2023)
- [When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?](https://mlsanity.com/paper/2309.05520/) (2023)
- [Prompting Is All You Need: Automated Android Bug Replay with Large Language Models](https://mlsanity.com/paper/2306.01987/) (2023)
- [Transformer-Based Language Models for Software Vulnerability Detection](https://mlsanity.com/paper/2204.03214/) (2022)
- ["I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data](https://mlsanity.com/paper/2212.05856/) (2022)
- [Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilities](https://mlsanity.com/paper/2402.17230/) (2024)
- [Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?](https://mlsanity.com/paper/2306.01754/) (2023)
- [LLbezpeky: Leveraging Large Language Models for Vulnerability Detection](https://mlsanity.com/paper/2401.01269/) (2024)
- [Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study](https://mlsanity.com/paper/2401.11105/) (2024)
- [An Empirical Study of Bots in Software Development -- Characteristics and Challenges from a Practitioner's Perspective](https://mlsanity.com/paper/2005.13969/) (2020)
- [Challenges and solutions when adopting DevSecOps: A systematic review](https://mlsanity.com/paper/2103.08266/) (2021)
- [Detecting Misuse of Security APIs: A Systematic Review](https://mlsanity.com/paper/2306.08869/) (2023)
- [Conversational DevBots for Secure Programming: An Empirical Study on SKF Chatbot](https://mlsanity.com/paper/2205.06200/) (2022)
- [Empirical Analysis of Software Vulnerabilities Causing Timing Side Channels](https://mlsanity.com/paper/2308.11862/) (2023)

## Built on by (most cited first, 2 of 2)

- [Software Vulnerability Management in the Era of Artificial Intelligence: An Industry Perspective](https://mlsanity.com/paper/2512.18261/) (2025)
- [Understanding the Impact of AI Code Assistants on Security API Usage: An Empirical Study](https://mlsanity.com/paper/2607.11348/) (2026)

## Abstract (arXiv)

Artificial Intelligence (AI) advancements have enabled the development of Large Language Models (LLMs) that can perform a variety of tasks with remarkable semantic understanding and accuracy. ChatGPT is one such LLM that has gained significant attention due to its impressive capabilities for assisting in various knowledge-intensive tasks. Due to the knowledge-intensive nature of engineering secure software, ChatGPT's assistance is expected to be explored for security-related tasks during the development/evolution of software. To gain an understanding of the potential of ChatGPT as an emerging technology for supporting software security, we adopted a two-fold approach. Initially, we performed an empirical study to analyse the perceptions of those who had explored the use of ChatGPT for security tasks and shared their views on Twitter. It was determined that security practitioners view ChatGPT as beneficial for various software security tasks, including vulnerability detection, information retrieval, and penetration testing. Secondly, we designed an experiment aimed at investigating the practicality of this technology when deployed as an oracle in real-world settings. In particular, we focused on vulnerability detection and qualitatively examined ChatGPT outputs for given prompts within this prominent software security task. Based on our analysis, responses from ChatGPT in this task are largely filled with generic security information and may not be appropriate for industry use. To prevent data leakage, we performed this analysis on a vulnerability dataset compiled after the OpenAI data cut-off date from real-world projects covering 40 distinct vulnerability types and 12 programming languages. We assert that the findings from this study would contribute to future research aimed at developing and evaluating LLMs dedicated to software security.