mlsanityData to 7 Oct 2026

Context-Augmented Code Generation: How Product Context Improves AI Coding Agent Decision Compliance by 49%

arXiv 2605.08112 ↗First posted 27 Apr 2026cs.SE cs.AI cs.CEcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

AI coding agents powered by large language models can read codebases and produce functional code, but they routinely violate team-specific product decisions that are invisible in the source code alone. We introduce a controlled benchmark measuring decision compliance, the rate at which an AI coding agent follows established product, design, and engineering decisions, across 8 realistic software engineering tasks containing 41 weighted decision points. We compare a baseline configuration (Claude Code with codebase access only) against an augmented configuration that adds Brief, a product-context retrieval system providing spec generation, mid-build consultation, and retrieval of recorded decisions, persona pain points, customer signals, and competitive intelligence. On identical prompts and the same repository, the augmented configuration achieves 95% decision compliance versus 46% for the baseline, a 49 percentage point improvement. Per-decision analysis reveals that the baseline achieves 100% compliance on decisions visible in the codebase and 0-33% on decisions requiring product context, suggesting that product-context retrieval is a key driver of the improvement. We release the benchmark repository, all 16 pull requests, and scoring harness for independent reproduction.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 18 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

14 of 18 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
  3. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges14 Jan

Other references · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
  10. 2022DocPrompting: Generating Code by Retrieving the Docscited by 151
  11. 2023Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPTcited by 71

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects
  2. 2026The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol
  3. 2026Using Agentic AI for contextualized and multifaceted code review at Ericsson
  4. 2025Agent READMEs: An Empirical Study of Context Files for Agentic Coding
  5. 2025Context Engineering for Multi-Agent LLM Code Assistants Using Elicit, NotebookLM, ChatGPT, and Claude Code
  6. 2026CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents
  7. 2026Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories
  8. 2025Context Engineering for AI Agents in Open-Source Software