# From Code Generation to Software Testing: AI Copilot with Context-Based RAG

arXiv: https://arxiv.org/abs/2504.01866 · HTML: https://mlsanity.com/paper/2504.01866/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 9 of 9)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Prompting Is Programming: A Query Language for Large Language Models](https://mlsanity.com/paper/2212.06094/) (2022)
- [Navigating the Complexity of Generative AI Adoption in Software Engineering](https://mlsanity.com/paper/2307.06081/) (2023)
- [Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language Models](https://mlsanity.com/paper/2307.14349/) (2023)
- [Ways of Applying Artificial Intelligence in Software Engineering](https://mlsanity.com/paper/1802.02033/) (2018)
- [Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models](https://mlsanity.com/paper/2503.15129/) (2025)
- [Defect Prediction Guided Search-Based Software Testing](https://mlsanity.com/paper/2109.12645/) (2021)
- [A Multi-Year Grey Literature Review on AI-assisted Test Automation](https://mlsanity.com/paper/2408.06224/) (2024)

## Built on by (most cited first, 2 of 2)

- [Future-Proofing Programmers: Optimal Knowledge Tracing for AI-Assisted Personalized Education](https://mlsanity.com/paper/2509.23996/) (2025)
- [From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Data](https://mlsanity.com/paper/2607.16845/) (2026)

## Abstract (arXiv)

The rapid pace of large-scale software development places increasing demands on traditional testing methodologies, often leading to bottlenecks in efficiency, accuracy, and coverage. We propose a novel perspective on software testing by positing bug detection and coding with fewer bugs as two interconnected problems that share a common goal, which is reducing bugs with limited resources. We extend our previous work on AI-assisted programming, which supports code auto-completion and chatbot-powered Q&A, to the realm of software testing. We introduce Copilot for Testing, an automated testing system that synchronizes bug detection with codebase updates, leveraging context-based Retrieval Augmented Generation (RAG) to enhance the capabilities of large language models (LLMs). Our evaluation demonstrates a 31.2% improvement in bug detection accuracy, a 12.6% increase in critical test coverage, and a 10.5% higher user acceptance rate, highlighting the transformative potential of AI-driven technologies in modern software development practices.