# Using Copilot Agent Mode to Automate Library Migration: A Quantitative Assessment

arXiv: https://arxiv.org/abs/2510.26699 · HTML: https://mlsanity.com/paper/2510.26699/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 30 Oct 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 6 of 6)

- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Migrating Code At Scale With LLMs At Google](https://mlsanity.com/paper/2504.09691/) (2025)
- [Automatic Library Migration Using Large Language Models: First Results](https://mlsanity.com/paper/2408.16151/) (2024)
- [Practical Considerations for Agentic LLM Systems](https://mlsanity.com/paper/2412.04093/) (2024)
- [Characterizing Python Library Migrations](https://mlsanity.com/paper/2207.01124/) (2022)
- [An Empirical Study of Python Library Migration Using Large Language Models](https://mlsanity.com/paper/2504.13272/) (2025)

## Built on by (most cited first, 3 of 3)

- [Configuration Smells in AGENTS.md Files: Common Mistakes in Configuring Coding Agents](https://mlsanity.com/paper/2606.15828/) (2026)
- [RepoRescue: An Empirical Study of LLM Agents on Whole-Repository Compatibility Rescue](https://mlsanity.com/paper/2607.01213/) (2026)
- [Understanding the Architecture of Coding Agents: An Exploratory Study Using a Research Prototype](https://mlsanity.com/paper/2608.10934/) (2026)

## Abstract (arXiv)

Keeping software systems up to date is essential to avoid technical debt, security vulnerabilities, and the rigidity typical of legacy systems. However, updating libraries and frameworks remains a time consuming and error-prone process. Recent advances in Large Language Models (LLMs) and agentic coding systems offer new opportunities for automating such maintenance tasks. In this paper, we evaluate the update of a well-known Python library, SQLAlchemy, across a dataset of ten client applications. For this task, we use the Github's Copilot Agent Mode, an autonomous AI systema capable of planning and executing multi-step migration workflows. To assess the effectiveness of the automated migration, we also introduce Migration Coverage, a metric that quantifies the proportion of API usage points correctly migrated. The results of our study show that the LLM agent was capable of migrating functionalities and API usages between SQLAlchemy versions (migration coverage: 100%, median), but failed to maintain the application functionality, leading to a low test-pass rate (39.75%, median).