# Investigating and Designing for Trust in AI-powered Code Generation Tools

arXiv: https://arxiv.org/abs/2305.11248 · HTML: https://mlsanity.com/paper/2305.11248/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2023.

## Built on by (most cited first, 30 of 34)

- [Agentic Much? Adoption of Coding Agents on GitHub](https://mlsanity.com/paper/2601.18341/) (2026)
- [Artificial Intelligence for Literature Reviews: Opportunities and Challenges](https://mlsanity.com/paper/2402.08565/) (2024)
- [A Study of LLMs' Preferences for Libraries and Programming Languages](https://mlsanity.com/paper/2503.17181/) (2025)
- [Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025](https://mlsanity.com/paper/2512.14012/) (2025)
- [BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks](https://mlsanity.com/paper/2404.07387/) (2024)
- [Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation](https://mlsanity.com/paper/2409.19182/) (2024)
- [Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations](https://mlsanity.com/paper/2407.08983/) (2024)
- [The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impaired](https://mlsanity.com/paper/2503.16491/) (2025)
- [Knowledge-Based Multi-Agent Framework for Automated Software Architecture Design](https://mlsanity.com/paper/2503.20536/) (2025)
- [Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articles](https://mlsanity.com/paper/2510.21293/) (2025)
- [Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity](https://mlsanity.com/paper/2601.18345/) (2026)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [A Transformer-Based Approach for Smart Invocation of Automatic Code Completion](https://mlsanity.com/paper/2405.14753/) (2024)
- [Generative AI Uses and Risks for Knowledge Workers in a Science Organization](https://mlsanity.com/paper/2501.16577/) (2025)
- [TableTalk: Scaffolding Spreadsheet Development with a Language Agent](https://mlsanity.com/paper/2502.09787/) (2025)
- [Human-AI Experience in Integrated Development Environments: A Systematic Literature Review](https://mlsanity.com/paper/2503.06195/) (2025)
- [Humans are Missing from AI Coding Agent Research](https://mlsanity.com/paper/2608.12355/) (2026)
- [Envisioning the Next-Generation AI Coding Assistants: Insights & Proposals](https://mlsanity.com/paper/2403.14592/) (2024)
- [CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs](https://mlsanity.com/paper/2510.11536/) (2025)
- [Tadashi: Enabling AI-Based Automated Code Generation With Guaranteed Correctness](https://mlsanity.com/paper/2410.03210/) (2024)
- [What You See Is Not Always What You Get: Evaluating GPT's Comprehension of Source Code](https://mlsanity.com/paper/2412.08098/) (2024)
- [Mapping Trustworthiness in Large Language Models: A Bibliometric Analysis Bridging Theory to Practice](https://mlsanity.com/paper/2503.04785/) (2025)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)
- [Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent](https://mlsanity.com/paper/2607.18257/) (2026)
- [Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectives](https://mlsanity.com/paper/2503.13793/) (2025)
- [Beyond Autocomplete: Designing CopilotLens Towards Transparent and Explainable AI Coding Agents](https://mlsanity.com/paper/2506.20062/) (2025)
- [Single Conversation Methodology: A Human-Centered Protocol for AI-Assisted Software Development](https://mlsanity.com/paper/2507.12665/) (2025)
- [Cracking CodeWhisperer: Analyzing Developers' Interactions and Patterns During Programming Tasks](https://mlsanity.com/paper/2510.11516/) (2025)
- [Do You Trust the Process?: Modeling Institutional Trust for Community Adoption of Reinforcement Learning Policies](https://mlsanity.com/paper/2510.22017/) (2025)
- [Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision](https://mlsanity.com/paper/2604.06723/) (2026)

## Abstract (arXiv)

As AI-powered code generation tools such as GitHub Copilot become popular, it is crucial to understand software developers' trust in AI tools -- a key factor for tool adoption and responsible usage. However, we know little about how developers build trust with AI, nor do we understand how to design the interface of generative AI systems to facilitate their appropriate levels of trust. In this paper, we describe findings from a two-stage qualitative investigation. We first interviewed 17 developers to contextualize their notions of trust and understand their challenges in building appropriate trust in AI code generation tools. We surfaced three main challenges -- including building appropriate expectations, configuring AI tools, and validating AI suggestions. To address these challenges, we conducted a design probe study in the second stage to explore design concepts that support developers' trust-building process by 1) communicating AI performance to help users set proper expectations, 2) allowing users to configure AI by setting and adjusting preferences, and 3) offering indicators of model mechanism to support evaluation of AI suggestions. We gathered developers' feedback on how these design concepts can help them build appropriate trust in AI-powered code generation tools, as well as potential risks in design. These findings inform our proposed design recommendations on how to design for trust in AI-powered code generation tools.