# Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs arXiv: https://arxiv.org/abs/2406.18294 · HTML: https://mlsanity.com/paper/2406.18294/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 37 of its 48 references that are in the map. - First posted on arXiv 26 Jun 2024. ## Builds on (most cited first, 30 of 37) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021) - [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023) - [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023) - [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023) - [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024) - [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022) - [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023) - [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024) - [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021) - [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022) - [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023) - [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023) - [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023) - [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022) - [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022) - [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023) - [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023) - [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024) - [RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems](https://mlsanity.com/paper/2306.03091/) (2023) - [LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://mlsanity.com/paper/2402.13753/) (2024) - [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022) - [OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement](https://mlsanity.com/paper/2402.14658/) (2024) - [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023) - [CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion](https://mlsanity.com/paper/2310.11248/) (2023) - [Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation](https://mlsanity.com/paper/2310.02304/) (2023) - [LLM Agents can Autonomously Hack Websites](https://mlsanity.com/paper/2402.06664/) (2024) - [Repository-Level Prompt Generation for Large Language Models of Code](https://mlsanity.com/paper/2206.12839/) (2022) - [Execution-Based Evaluation for Open-Domain Code Generation](https://mlsanity.com/paper/2212.10481/) (2022) ## Built on by (most cited first, 13 of 13) - [SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents](https://mlsanity.com/paper/2601.16746/) (2026) - [Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches](https://mlsanity.com/paper/2510.04905/) (2025) - [Understanding and Enhancing the Transferability of Jailbreaking Attacks](https://mlsanity.com/paper/2502.03052/) (2025) - [Continual Multimodal Contrastive Learning](https://mlsanity.com/paper/2503.14963/) (2025) - [In Line with Context: Repository-Level Code Generation via Context Inlining](https://mlsanity.com/paper/2601.00376/) (2026) - [L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models](https://mlsanity.com/paper/2505.17505/) (2025) - [OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis](https://mlsanity.com/paper/2501.04561/) (2025) - [Exploring the Challenges and Opportunities of AI-assisted Codebase Generation](https://mlsanity.com/paper/2508.07966/) (2025) - [SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion](https://mlsanity.com/paper/2510.17925/) (2025) - [FDC: Fast KV Dimensionality Compression for Efficient LLM Inference](https://mlsanity.com/paper/2408.04107/) (2024) - [MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion benchmark](https://mlsanity.com/paper/2601.03708/) (2026) - [Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory](https://mlsanity.com/paper/2608.25570/) (2026) - [VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks](https://mlsanity.com/paper/2609.23570/) (2026) ## Abstract (arXiv) Some recently developed code large language models (Code LLMs) have been pre-trained on repository-level code data (Repo-Code LLMs), enabling these models to recognize repository structures and utilize cross-file information for code completion. However, in real-world development scenarios, simply concatenating the entire code repository often exceeds the context window limits of these Repo-Code LLMs, leading to significant performance degradation. In this study, we conducted extensive preliminary experiments and analyses on six Repo-Code LLMs. The results indicate that maintaining the topological dependencies of files and increasing the code file content in the completion prompts can improve completion accuracy; pruning the specific implementations of functions in all dependent files does not significantly reduce the accuracy of completions. Based on these findings, we proposed a strategy named Hierarchical Context Pruning (HCP) to construct completion prompts with high informational code content. The HCP models the code repository at the function level, maintaining the topological dependencies between code files while removing a large amount of irrelevant code content, significantly reduces the input length for repository-level code completion. We applied the HCP strategy in experiments with six Repo-Code LLMs, and the results demonstrate that our proposed method can significantly enhance completion accuracy while substantially reducing the length of input. Our code and data are available at https://github.com/Hambaobao/HCP-Coder.