Can Agents Design Libraries for Agents?
Not on the map. Why?
Abstract (arXiv)
Agents increasingly build on code written by other agents, and they reimplement rather than reuse, growing the codebases later agents must work in. To measure how well agents design libraries for other agents, we introduce LibraryDesignBench, a two-phase benchmark in which an agent implements a full-featured library from a specification that defines required capabilities and potential use cases without prescribing the design. We evaluate the library through the correctness and simplicity of programs written by three user agents from different model families. The benchmark spans 242 expert-validated programming problems across 15 library-design tasks in four languages. On eleven of the fifteen tasks, agent designers reproduce the abstractions of the human-written production library. Downstream agents adopt agent- and human-written libraries alike but underuse them, reimplementing capabilities the library already provides. Our failure analysis finds that downstream agents write extra code mainly because agent-written libraries are rigid or hard to use, not because capabilities are missing. We also experiment with giving designers more prescriptive, agent-first guidance and having them test their library with subagents; this improves downstream scores and yields simpler programs. LibraryDesignBench provides both a testbed for evaluating library-design practices for agent users and an initial design baseline that improves downstream reuse.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 65 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
46 of 65 references.
References · 46
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
- 2023Large Language Models as Tool Makerscited by 293
- 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 130
- 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 122
- 2026Kimi K3: Open Frontier Intelligencecited by 99
- 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Modelscited by 90
- 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
34 more
- 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agentscited by 56
- 2022When Language Model Meets Private Librarycited by 54
- 2024Commit0: Library Generation from Scratchcited by 50
- 2026ProgramBench: Can Language Models Rebuild Programs From Scratch?cited by 47
- 2026DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Taskscited by 41
- 2024ReGAL: Refactoring Programs to Discover Generalizable Abstractionscited by 32
- 2024LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completioncited by 29
- 2025RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Codecited by 28
- 2025ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generationcited by 26
- 2026SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Taskscited by 24
- 2025Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projectscited by 23
- 2024On Mitigating Code LLM Hallucinations with API Documentationcited by 21
- 2025A Study of LLMs' Preferences for Libraries and Programming Languagescited by 20
- 2025When LLMs Meet API Documentation: Can Retrieval Augmentation Aid Code Generation Just as It Helps Developers?cited by 12
- 2026RepoZero: Can LLMs Generate a Code Repository from Scratch?cited by 12
- 2026Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environmentscited by 11
- 2026CodeTaste: Can LLMs Generate Human-Level Code Refactorings?cited by 8
- 2026DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratchcited by 6
- 2025ShapeLib: Designing a library of programmatic 3D shape abstractions with Large Language Modelscited by 5
- 2026What to Cut? Predicting Unnecessary Methods in Agentic Code Generationcited by 5
- 2026Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarioscited by 5
- 2026ICAE-Bench: Evaluating Coding Agents as Interactive Project Builderscited by 5
- 2025Refactoring Codebases through Library Designcited by 4
- 2026Rethinking Code Complexity Through the Lens of Large Language Modelscited by 3
- 2026Building to the Test: Coding Agents Deliver What You Check, Not What You Requestedcited by 3
- 2026Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metricscited by 2
- 2026A Survey on Large Language Model Impact on Software Evolvability and Maintainability: the Good, the Bad, the Ugly, and the Remedycited by 2
- 2026ADK Arena: Evaluating Agent Development Kits via LLM-as-a-Developercited by 2
- 2026Repo0: Design-Driven Zero-to-All Code Generationcited by 2
- 2025A Study of Library Usage in Agent-Authored Pull Requestscited by 1
- 2026From Human Interfaces to Agent Interfaces: Rethinking Software Design in the Age of AI-Native Systemscited by 1
- 2026Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agentscited by 1
- 2026Does Code Cleanliness Affect Coding Agents? A Controlled Minimal-Pair Studycited by 1
- 2026Is Agent Code Less Maintainable Than Human Code?cited by 1
Built on by
Not cited yet.