Building an Internal Coding Agent at Zup: Lessons and Open Questions
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Enterprise teams building internal coding agents face a gap between prototype performance and production readiness. The root cause is that technical model quality alone is insufficient -- tool design, safety enforcement, state management, and human trust calibration are equally decisive, yet underreported in the literature. We present CodeGen, an internal coding agent at Zup, and show that targeted tool design (e.g., string-replacement edits over full-file rewrites) and layered safety guardrails improved agent reliability more than prompt engineering, while progressive human oversight modes drove organic adoption without mandating trust. These findings suggest that the engineering decisions surrounding the model -- not the model itself -- determine whether a coding agent delivers real value in practice.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 13 of its 16 references that are in the map. First posted on arXiv 10 Apr 2026.
Builds on
13 of 16 references.
Closest ancestors · 2
Same direction, newest first
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
- 2024CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology2 Feb
Other references · 11
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2022A Systematic Evaluation of Large Language Models of Codecited by 360
- 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
- 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
- 2023Lessons from Building StackSpot AI: A Contextualized AI Coding Assistantcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
- 2026Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
- 2026SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- 2026An Empirical Study of Harness Design for Coding Agents
- 2026Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
- 2026Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction
- 2026A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents
- 2026The End of Code Review: Coding Agents Supersede Human Inspection