# Assessing AI-Based Code Assistants in Method Generation Tasks

arXiv: https://arxiv.org/abs/2402.09022 · HTML: https://mlsanity.com/paper/2402.09022/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2024.

## Built on by (most cited first, 5 of 5)

- [AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation](https://mlsanity.com/paper/2504.04220/) (2025)
- [Disrupting Test Development with AI Assistants](https://mlsanity.com/paper/2411.02328/) (2024)
- [Paradigm shift on Coding Productivity Using GenAI](https://mlsanity.com/paper/2504.18404/) (2025)
- [Beyond Autocomplete: Designing CopilotLens Towards Transparent and Explainable AI Coding Agents](https://mlsanity.com/paper/2506.20062/) (2025)
- [Generation of Web Apps with Agentic IDEs: An Empirical Assessment](https://mlsanity.com/paper/2608.20903/) (2026)

## Abstract (arXiv)

AI-based code assistants are increasingly popular as a means to enhance productivity and improve code quality. This study compares four AI-based code assistants, GitHub Copilot, Tabnine, ChatGPT, and Google Bard, in method generation tasks, assessing their ability to produce accurate, correct, and efficient code. Results show that code assistants are useful, with complementary capabilities, although they rarely generate ready-to-use correct code.