# Are We Ready to Embrace Generative AI for Software Q&A?

arXiv: https://arxiv.org/abs/2307.09765 · HTML: https://mlsanity.com/paper/2307.09765/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2023.

## Built on by (most cited first, 5 of 5)

- [Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps](https://mlsanity.com/paper/2309.04142/) (2023)
- [Just another copy and paste? Comparing the security vulnerabilities of ChatGPT generated code and StackOverflow answers](https://mlsanity.com/paper/2403.15600/) (2024)
- [Generative AI and Empirical Software Engineering: A Paradigm Shift](https://mlsanity.com/paper/2502.08108/) (2025)
- [LEGION: Harnessing Pre-trained Language Models for GitHub Topic Recommendations with Distribution-Balance Loss](https://mlsanity.com/paper/2403.05873/) (2024)
- [Human-Aligned Enhancement of Programming Answers with LLMs Guided by User Feedback](https://mlsanity.com/paper/2601.17604/) (2026)

## Abstract (arXiv)

Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, specifically by 10% on the overall score. We release the data, analysis scripts, and detailed results at https://anonymous.4open.science/r/GAI4SQA-FD5C.