mlsanityData to 7 Oct 2026

Pragmatic Reasoning improves LLM Code Generation

arXiv 2502.15835 ↗First posted 20 Feb 2025cs.CL cs.AI cs.SEcited by 6

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Pragmatic reasoning helps interlocutors infer intended meaning from ambiguous or underspecified messages by considering shared context and counterfactual alternatives. Similar challenges arise in natural language-to-code generation, where user instructions often admit multiple plausible candidate programs. However, direct RSA-style inference is difficult because it requires probability estimation over large spaces of programs and alternative instructions. We propose CodeRSA, an RSA-motivated reranking method that makes pragmatic reasoning tractable through local pragmatic contests among sampled code candidates. CodeRSA constructs candidate-induced alternative instructions and estimates which candidates are most distinctively supported by the original instruction, avoiding global normalization over the full program-instruction space. We evaluate CodeRSA on HumanEval+, MBPP+, and BigCodeBench using four open-weight instruction-following models. CodeRSA achieves the strongest average accuracy in 10 of 12 model-benchmark settings and remains competitive in the remaining cases. Further analyses show that its gains come from combining local pairwise pragmatic comparison with broader global support, suggesting a scalable direction for language-to-code reranking under natural-language uncertainty.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 14 of its 35 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

14 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Qwen2.5-Coder Technical Report18 Sep

Other references · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  6. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  7. 2022CodeT: Code Generation with Generated Testscited by 369
  8. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  9. 2016Latent Predictor Networks for Code Generationcited by 73
  10. 2023Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPTcited by 71
  11. 2016Mutual Information and Diverse Decoding Improve Neural Machine Translationcited by 35
  12. 2023AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisationcited by 31
1 more
  1. 2024Analysis on LLMs Performance for Code Summarizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset
  2. 2026Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
  3. 2026Pseudo2CodeQA: A Benchmark for LLM-Based Structured Algorithmic Reasoning in Code Generation
  4. 2025CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation
  5. 2025PACIFIC: a framework for generating benchmarks to check Precise Automatically Checked Instruction Following In Code
  6. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
  7. 2025Open the Oyster: Empirical Evaluation and Improvement of Code Reasoning Confidence in LLMs
  8. 2026Code Is More Than Text: Uncertainty Estimation for Code Generation