# All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs

arXiv: https://arxiv.org/abs/2205.10692 · HTML: https://mlsanity.com/paper/2205.10692/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 48 references that are in the map.
- First posted on arXiv 21 May 2022.

## Builds on (most cited first, 11 of 11)

- [IntelliCode Compose: Code Generation Using Transformer](https://mlsanity.com/paper/2005.08025/) (2020)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [Pythia: AI-assisted Code Completion System](https://mlsanity.com/paper/1912.00742/) (2019)
- [CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences](https://mlsanity.com/paper/2202.06689/) (2022)
- [Fast and Memory-Efficient Neural Code Completion](https://mlsanity.com/paper/2004.13651/) (2020)
- [Learning Autocompletion from Real-World Datasets](https://mlsanity.com/paper/2011.04542/) (2020)
- [To What Extent do Deep Learning-based Code Recommenders Generate Predictions by Cloning Code from the Training Set?](https://mlsanity.com/paper/2204.06894/) (2022)
- [Sequence Model Design for Code Completion in the Modern IDE](https://mlsanity.com/paper/2004.05249/) (2020)
- [Towards Full-line Code Completion with Neural Language Models](https://mlsanity.com/paper/2009.08603/) (2020)
- [Recommendation of Move Method Refactoring Using Path-Based Representation of Code](https://mlsanity.com/paper/2002.06392/) (2020)
- [The IntelliJ Platform: a Framework for Building Plugins and Mining Software Data](https://mlsanity.com/paper/2110.00141/) (2021)

## Built on by (most cited first, 9 of 9)

- [Language Models for Code Completion: A Practical Evaluation](https://mlsanity.com/paper/2402.16197/) (2024)
- [From Commit Message Generation to History-Aware Commit Message Completion](https://mlsanity.com/paper/2308.07655/) (2023)
- [Towards an Understanding of Context Utilization in Code Intelligence](https://mlsanity.com/paper/2504.08734/) (2025)
- [Full Line Code Completion: Bringing AI to Desktop](https://mlsanity.com/paper/2405.08704/) (2024)
- [Understanding and supporting how developers prompt for LLM-powered code editing in practice](https://mlsanity.com/paper/2504.20196/) (2025)
- [WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching](https://mlsanity.com/paper/2601.11652/) (2026)
- [FDI: Attack Neural Code Generation Systems through User Feedback Channel](https://mlsanity.com/paper/2408.04194/) (2024)
- [Structure-Aware Corpus Construction and User-Perception-Aligned Metrics for Large-Language-Model Code Completion](https://mlsanity.com/paper/2505.13073/) (2025)
- [TreeRanker: Fast and Model-agnostic Ranking System for Code Suggestions in IDEs](https://mlsanity.com/paper/2508.02455/) (2025)

## Abstract (arXiv)

In this work, we propose an approach for collecting completion usage logs from the users in an IDE and using them to train a machine learning based model for ranking completion candidates. We developed a set of features that describe completion candidates and their context, and deployed their anonymized collection in the Early Access Program of IntelliJ-based IDEs. We used the logs to collect a dataset of code completions from users, and employed it to train a ranking CatBoost model. Then, we evaluated it in two settings: on a held-out set of the collected completions and in a separate A/B test on two different groups of users in the IDE. Our evaluation shows that using a simple ranking model trained on the past user behavior logs significantly improved code completion experience. Compared to the default heuristics-based ranking, our model demonstrated a decrease in the number of typing actions necessary to perform the completion in the IDE from 2.073 to 1.832. The approach adheres to privacy requirements and legal constraints, since it does not require collecting personal information, performing all the necessary anonymization on the client's side. Importantly, it can be improved continuously: implementing new features, collecting new data, and evaluating new models - this way, we have been using it in production since the end of 2020.