# LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion

arXiv: https://arxiv.org/abs/2406.09834 · HTML: https://mlsanity.com/paper/2406.09834/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2024.

## Built on by (most cited first, 30 of 32)

- [Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects](https://mlsanity.com/paper/2511.04427/) (2025)
- [ReCode: Updating Code API Knowledge with Reinforcement Learning](https://mlsanity.com/paper/2506.20495/) (2025)
- [RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation](https://mlsanity.com/paper/2503.16922/) (2025)
- [GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models](https://mlsanity.com/paper/2411.05830/) (2024)
- [CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval](https://mlsanity.com/paper/2506.11066/) (2025)
- [Rethinking Autonomy: Preventing Failures in AI-Driven Software Engineering](https://mlsanity.com/paper/2508.11824/) (2025)
- [Environment-Aware Code Generation: How far are We?](https://mlsanity.com/paper/2601.12262/) (2026)
- [GitChameleon 2.0: Evaluating AI Code Generation Against Python Library Version Incompatibilities](https://mlsanity.com/paper/2507.12367/) (2025)
- [An Empirical Study of OpenAI API Discussions on Stack Overflow](https://mlsanity.com/paper/2505.04084/) (2025)
- [Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries](https://mlsanity.com/paper/2509.22202/) (2025)
- [Understanding Robustness of Model Editing in Code LLMs](https://mlsanity.com/paper/2511.03182/) (2025)
- [A Study of Library Usage in Agent-Authored Pull Requests](https://mlsanity.com/paper/2512.11589/) (2025)
- [TimeMachine-bench: A Benchmark for Evaluating Model Capabilities in Repository-Level Migration Tasks](https://mlsanity.com/paper/2601.22597/) (2026)
- [Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility](https://mlsanity.com/paper/2602.07195/) (2026)
- [KCoEvo: A Knowledge Graph Augmented Framework for Evolutionary Code Generation](https://mlsanity.com/paper/2603.07581/) (2026)
- [The Shrinking Lifespan of LLMs in Science](https://mlsanity.com/paper/2604.07530/) (2026)
- [Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code](https://mlsanity.com/paper/2605.05267/) (2026)
- [Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions](https://mlsanity.com/paper/2605.06279/) (2026)
- [Characterizing Readability Issue Patterns and the Role of Prompt Design in LLM-Generated Code](https://mlsanity.com/paper/2605.13280/) (2026)
- [The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation](https://mlsanity.com/paper/2606.04057/) (2026)
- [Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs](https://mlsanity.com/paper/2606.30810/) (2026)
- [SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements](https://mlsanity.com/paper/2607.27409/) (2026)
- [Toward Executable Repository-Level Code Generation via Environment Alignment](https://mlsanity.com/paper/2604.03622/) (2026)
- [Stdlib or Third-Party? Empirical Performance and Correctness of LLM-Assisted Zero-Dependency Python Libraries](https://mlsanity.com/paper/2605.21405/) (2026)
- [Beyond Pass Rate: A Multilingual, Execution-Grounded Evaluation of Open Code LLMs](https://mlsanity.com/paper/2606.08840/) (2026)
- [LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models](https://mlsanity.com/paper/2606.25402/) (2026)
- [The Hitchhiker's Guide to Monoculture: AI Homogenizes Syntax, Not (Necessarily) Semantics](https://mlsanity.com/paper/2607.13077/) (2026)
- [Set-shifting Behavioral Test for Harnessed Agents](https://mlsanity.com/paper/2607.13396/) (2026)
- [Characterizing the Quality Profile of AI-Generated C++ in Production](https://mlsanity.com/paper/2608.06640/) (2026)
- [What Was Once Learned May Need to Be Unlearned: Machine Unlearning for Deprecated API Knowledge in Large Language Models](https://mlsanity.com/paper/2609.25786/) (2026)

## Abstract (arXiv)

Large language models (LLMs), pre-trained or fine-tuned on large code corpora, have shown effectiveness in generating code completions. However, in LLM-based code completion, LLMs may struggle to use correct and up-to-date Application Programming Interfaces (APIs) due to the rapid and continuous evolution of libraries. While existing studies have highlighted issues with predicting incorrect APIs, the specific problem of deprecated API usage in LLM-based code completion has not been thoroughly investigated. To address this gap, we conducted the first evaluation study on deprecated API usage in LLM-based code completion. This study involved seven advanced LLMs, 145 API mappings from eight popular Python libraries, and 28,125 completion prompts. The study results reveal the status quo (i.e., API usage plausibility and deprecated usage rate) of deprecated API and replacing API usage in LLM-based code completion from the perspectives of model, prompt, and library, and indicate the root causes behind. Based on these findings, we propose two lightweight fixing approaches, REPLACEAPI and INSERTPROMPT, which can serve as baseline approaches for future research on mitigating deprecated API usage in LLM-based completion. Additionally, we provide implications for future research on integrating library evolution with LLM-driven software development.