# Trust Calibration in IDEs: Paving the Way for Widespread Adoption of AI Refactoring

arXiv: https://arxiv.org/abs/2412.15948 · HTML: https://mlsanity.com/paper/2412.15948/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 20 Dec 2024.

## Builds on (most cited first, 13 of 13)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Large Language Models for Software Engineering: Survey and Open Problems](https://mlsanity.com/paper/2310.03533/) (2023)
- [An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation](https://mlsanity.com/paper/1812.08693/) (2018)
- [Examining Zero-Shot Vulnerability Repair with Large Language Models](https://mlsanity.com/paper/2112.02125/) (2021)
- [Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?](https://mlsanity.com/paper/2204.04741/) (2022)
- [Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System](https://mlsanity.com/paper/2204.07874/) (2022)
- ["It would work for me too": How Online Communities Shape Software Developers' Trust in AI-Powered Code Generation Tools](https://mlsanity.com/paper/2212.03491/) (2022)
- [Towards Generating Functionally Correct Code Edits from Natural Language Issue Descriptions](https://mlsanity.com/paper/2304.03816/) (2023)
- [Don't Disturb Me: Challenges of Interacting with SoftwareBots on Open Source Software Projects](https://mlsanity.com/paper/2103.13950/) (2021)
- [Code Red: The Business Impact of Code Quality -- A Quantitative Study of 39 Proprietary Production Codebases](https://mlsanity.com/paper/2203.04374/) (2022)
- [Performance Analysis of Out-of-Distribution Detection on Trained Neural Networks](https://mlsanity.com/paper/2204.12378/) (2022)
- [On the Unhappiness of Software Developers](https://mlsanity.com/paper/1703.04993/) (2017)
- [Mutation Testing Optimisations using the Clang Front-end](https://mlsanity.com/paper/2210.17215/) (2022)

## Built on by (most cited first, 1 of 1)

- [Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectives](https://mlsanity.com/paper/2503.13793/) (2025)

## Abstract (arXiv)

In the software industry, the drive to add new features often overshadows the need to improve existing code. Large Language Models (LLMs) offer a new approach to improving codebases at an unprecedented scale through AI-assisted refactoring. However, LLMs come with inherent risks such as braking changes and the introduction of security vulnerabilities. We advocate for encapsulating the interaction with the models in IDEs and validating refactoring attempts using trustworthy safeguards. However, equally important for the uptake of AI refactoring is research on trust development. In this position paper, we position our future work based on established models from research on human factors in automation. We outline action research within CodeScene on development of 1) novel LLM safeguards and 2) user interaction that conveys an appropriate level of trust. The industry collaboration enables large-scale repository analysis and A/B testing to continuously guide the design of our research interventions.