From Guessing to Placeholding: A Cost-Theoretic Framework for Uncertainty-Aware Code Completion
Not on the map. Why?
Abstract (arXiv)
While Large Language Models (LLMs) have demonstrated exceptional proficiency in code completion, they typically adhere to a Hard Completion (HC) paradigm, compelling the generation of fully concrete code even amidst insufficient context. Our analysis of 3 million real-world interactions exposes the limitations of this strategy: 61% of the generated suggestions were either edited after acceptance or rejected despite exhibiting over 80% similarity to the user's subsequent code, suggesting that models frequently make erroneous predictions at specific token positions. Motivated by this observation, we propose Adaptive Placeholder Completion (APC), a collaborative framework that extends HC by strategically outputting explicit placeholders at high-entropy positions, allowing users to fill directly via IDE navigation. Theoretically, we formulate code completion as a cost-minimization problem under uncertainty. Premised on the observation that filling placeholders incurs lower cost than correcting errors, we prove the existence of a critical entropy threshold above which APC achieves strictly lower expected cost than HC. We instantiate this framework by constructing training data from filtered real-world edit logs and design a cost-based reward function for reinforcement learning. Extensive evaluations across 1.5B--14B parameter models demonstrate that APC reduces expected editing costs from 19% to 50% while preserving standard HC performance. Our work provides both a theoretical foundation and a practical training framework for uncertainty-aware code completion, demonstrating that adaptive abstention can be learned end-to-end without sacrificing conventional completion quality.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 37 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
34 of 37 references.
References · 34
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Code Llama: Open Foundation Models for Codecited by 2,105
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2024Qwen2.5-Coder Technical Reportcited by 1,151
22 more
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
- 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023StarCoder: may the source be with you!cited by 790
- 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
- 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
- 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 357
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
- 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
- 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
- 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
- 2022Efficient Training of Language Models to Fill in the Middlecited by 248
- 2020IntelliCode Compose: Code Generation Using Transformercited by 225
- 2024CodeGemma: Open Code Models Based on Gemmacited by 145
- 2022Productivity Assessment of Neural Code Completioncited by 97
- 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
- 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
- 2023Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repaircited by 55
- 2024Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinationscited by 51
- 2023SkCoder: A Sketch-based Approach for Automatic Code Generationcited by 40
- 2023Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generationcited by 27
Built on by
Not cited yet.