mlsanityData to 7 Oct 2026

The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context

arXiv 2608.14838 ↗First posted 14 Aug 2026cs.SE cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Retrieval components for code assistants are tuned against retrieval metrics: a configuration that raises recall@k is adopted, and downstream task success is assumed to follow. We report a controlled case study in code repair, not a new phenomenon but a deployed-flag, execution-graded instance of the known relevance-diversity and objective-mismatch tradeoff (Levy et al., 2025). On SWE-bench Verified we inject a retriever's hits as a fixed 12-slot context pack with no search tools and toggle one flag (one-chunk-per-file deduplication) on an otherwise identical stack. The flag is the higher-recall configuration (gold file present in 0.878 of served packs against 0.806 disabled), yet disabling it, trading file breadth for within-file depth, raises the single-shot resolve rate: gpt-5.6-sol +7.6pp (39.2% to 46.8%, n=500, McNemar exact p=0.0003), and a pre-registered open-weights replication any reviewer can re-run (Qwen3.6-27B, +3.6pp, n=499, p=0.0133); both survive repository-clustered inference. The gain tracks within-file anchor dose, and a random-chunk control refutes an argmax-selection artifact. We map where it holds: it reverses on a lexical BM25 retriever (-3.2pp, significant cross-paradigm interaction), is not detected under unrestricted-Read agents (a powered null), and across four languages (SWE-PolyBench, N=617) is positive but not significant (+2.6pp, p=0.056), a mapped boundary rather than a confirmed extension. Operationally, at a tight fixed budget: do not hard-deduplicate by file, and A/B packing policies against the task, not the metric the flag was tuned to.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

26 of 36 references.

References · 26

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  6. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 118
  7. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 111
  8. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generationcited by 90
  9. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  10. 2025SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agentscited by 80
  11. 2024Repoformer: Selective Retrieval for Repository-Level Code Completioncited by 55
  12. 2020PARADE: Passage Representation Aggregation for Document Rerankingcited by 44
14 more
  1. 2026ContextBench: A Benchmark for Context Retrieval in Coding Agentscited by 23
  2. 2025More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAGcited by 16
  3. 2026Compressing Code Context for LLM-based Issue Resolutioncited by 12
  4. 2026RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Modelscited by 5
  5. 2026FastContext: Training Efficient Repository Explorer for Coding Agentscited by 4
  6. 2025Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgetscited by 3
  7. 2026On the Role of Fault Localization Context for LLM-Based Program Repaircited by 3
  8. 2026Byte-Exact Deduplication in Retrieval-Augmented Generation: A Three-Regime Empirical Analysis Across Public Benchmarkscited by 3
  9. 2025When More Retrieval Hurts: Retrieval-Augmented Code Review Generationcited by 2
  10. 2026Scaling Retrieval Augmented Generation with RAG Fusion: Lessons from an Industry Deploymentcited by 2
  11. 2026What Context Does a Coding Agent Actually Need to Act?cited by 2
  12. 2026The Limits of Long-Context Reasoning in Automated Bug Fixingcited by 1
  13. 2026DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understandingcited by 1
  14. 2026The Powerless Noise: How Experimental Settings Shape the Reported Power of Noisecited by 1

Built on by

Not cited yet.