# Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes

arXiv: https://arxiv.org/abs/2311.18452 · HTML: https://mlsanity.com/paper/2311.18452/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2023.

## Built on by (most cited first, 9 of 9)

- [How AI Impacts Skill Formation](https://mlsanity.com/paper/2601.20245/) (2026)
- [LLMs' Reshaping of People, Processes, Products, and Society in Software Development: A Comprehensive Exploration with Early Adopters](https://mlsanity.com/paper/2503.05012/) (2025)
- [The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study](https://mlsanity.com/paper/2507.03156/) (2025)
- [Exploring the Challenges and Opportunities of AI-assisted Codebase Generation](https://mlsanity.com/paper/2508.07966/) (2025)
- [Making Sense of AI Limitations: How Individual Perceptions Shape Organizational Readiness for AI Adoption](https://mlsanity.com/paper/2502.15870/) (2025)
- [Counterfactual-based Agent Influence Ranker for Agentic AI Workflows](https://mlsanity.com/paper/2510.25612/) (2025)
- [The State of Generative AI in Software Development: Insights from Literature and a Developer Survey](https://mlsanity.com/paper/2603.16975/) (2026)
- [Contextualized AI for Cyber Defense: An Automated Survey using LLMs](https://mlsanity.com/paper/2409.13524/) (2024)
- [From Junior to Senior: Allocating Agency and Navigating Professional Growth in Agentic AI-Mediated Software Engineering](https://mlsanity.com/paper/2602.00496/) (2026)

## Abstract (arXiv)

In the rapidly advancing field of artificial intelligence, software development has emerged as a key area of innovation. Despite the plethora of general-purpose AI assistants available, their effectiveness diminishes in complex, domain-specific scenarios. Noting this limitation, both the academic community and industry players are relying on contextualized coding AI assistants. These assistants surpass general-purpose AI tools by integrating proprietary, domain-specific knowledge, offering precise and relevant solutions. Our study focuses on the initial experiences of 62 participants who used a contextualized coding AI assistant -- named StackSpot AI -- in a controlled setting. According to the participants, the assistants' use resulted in significant time savings, easier access to documentation, and the generation of accurate codes for internal APIs. However, challenges associated with the knowledge sources necessary to make the coding assistant access more contextual information as well as variable responses and limitations in handling complex codes were observed. The study's findings, detailing both the benefits and challenges of contextualized AI assistants, underscore their potential to revolutionize software development practices, while also highlighting areas for further refinement.