# Evaluating Privacy Questions From Stack Overflow: Can ChatGPT Compete?

arXiv: https://arxiv.org/abs/2306.11174 · HTML: https://mlsanity.com/paper/2306.11174/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 19 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Is ChatGPT the Ultimate Programming Assistant -- How far is it?](https://mlsanity.com/paper/2304.11938/) (2023)
- [ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design](https://mlsanity.com/paper/2303.07839/) (2023)
- [Generative Adversarial Network for Abstractive Text Summarization](https://mlsanity.com/paper/1711.09357/) (2017)
- [Comparing Software Developers with ChatGPT: An Empirical Investigation](https://mlsanity.com/paper/2305.11837/) (2023)
- [Engineering Privacy by Design: Are engineers ready to live up to the challenge?](https://mlsanity.com/paper/2006.04579/) (2020)
- [A Mixed-method Study on Security and Privacy Practices in Danish Companies](https://mlsanity.com/paper/2104.04030/) (2021)
- ["Money makes the world go around'': Identifying Barriers to Better Privacy in Children's Apps From Developers' Perspectives](https://mlsanity.com/paper/2111.14596/) (2021)
- [Understanding Developers Privacy Concerns Through Reddit Thread Analysis](https://mlsanity.com/paper/2304.07650/) (2023)

## Built on by (most cited first, 2 of 2)

- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [Using LLMs in Generating Design Rationale for Software Architecture Decisions](https://mlsanity.com/paper/2504.20781/) (2025)

## Abstract (arXiv)

Stack Overflow and other similar forums are used commonly by developers to seek answers for their software development as well as privacy-related concerns. Recently, ChatGPT has been used as an alternative to generate code or produce responses to developers' questions. In this paper, we aim to understand developers' privacy challenges by evaluating the types of privacy-related questions asked on Stack Overflow. We then conduct a comparative analysis between the accepted responses given by Stack Overflow users and the responses produced by ChatGPT for those extracted questions to identify if ChatGPT could serve as a viable alternative. Our results show that most privacy-related questions are related to choice/consent, aggregation, and identification. Furthermore, our findings illustrate that ChatGPT generates similarly correct responses for about 56% of questions, while for the rest of the responses, the answers from Stack Overflow are slightly more accurate than ChatGPT.