Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape
Not on the map. Why?
Abstract (arXiv)
Context: Software engineering (SE) researchers increasingly study Generative AI (GenAI) while also incorporating it into their own research practices. Despite rapid adoption, there is limited empirical evidence on how GenAI is used in SE research and its implications for research practices and governance. Aims: We conduct a large-scale survey of 457 SE researchers publishing in top venues between 2023 and 2025. Method: Using quantitative and qualitative analyses, we examine who uses GenAI and why, where it is used across research activities, and how researchers perceive its benefits, opportunities, challenges, risks, and governance. Results: GenAI use is widespread, with many researchers reporting pressure to adopt and align their work with it. Usage is concentrated in writing and early-stage activities, while methodological and analytical tasks remain largely human-driven. Although productivity gains are widely perceived, concerns about trust, correctness, and regulatory uncertainty persist. Researchers highlight risks such as inaccuracies and bias, emphasize mitigation through human oversight and verification, and call for clearer governance, including guidance on responsible use and peer review. Conclusion: We provide a fine-grained, SE-specific characterization of GenAI use across research activities, along with taxonomies of GenAI use cases for research and peer review, opportunities, risks, mitigation strategies, and governance needs. These findings establish an empirical baseline for the responsible integration of GenAI into academic practice.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 57 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
19 of 57 references.
References · 19
- 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
- 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 200
- 2025LLM4SR: A Survey on Large Language Models for Scientific Researchcited by 83
- 2024Delving into LLM-assisted writing in biomedical publications through excess vocabularycited by 78
- 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 42
- 2023Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languagescited by 36
- 2024AI for social science and social science of AI: A Surveycited by 29
- 2020Sampling in Software Engineering Research: A Critical Review and Guidelinescited by 23
- 2024Towards Evaluation Guidelines for Empirical Studies involving LLMscited by 15
- 2024The Promise and Challenges of Using LLMs to Accelerate the Screening Process of Systematic Reviewscited by 14
- 2025Generative AI and Empirical Software Engineering: A Paradigm Shiftcited by 9
- 2024Applications and Implications of Large Language Models in Qualitative Analysis: A New Frontier for Empirical Software Engineeringcited by 8
7 more
- 2025Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Researchcited by 5
- 2025LLM-Assisted Thematic Analysis: Opportunities, Limitations, and Recommendationscited by 5
- 2024What Guides Our Choices? Modeling Developers' Trust and Behavioral Intentions Towards GenAIcited by 3
- 2025Empirical and Sustainability Aspects of Software Engineering Research in the Era of Large Language Models: A Reflectioncited by 3
- 2021"How Was Your Weekend?" Software Development Teams Working From Home During COVID-19cited by 2
- 2025OLAF: Towards Robust LLM-Based Annotation Framework in Empirical Software Engineeringcited by 1
- 2025An Investigation on How AI-Generated Responses Affect SoftwareEngineering Surveyscited by 1
Built on by
Not cited yet.