# Mitigating Errors in LLM-Generated Web API Invocations via Retrieval-Augmented Generation and Constrained Decoding

arXiv: https://arxiv.org/abs/2607.05936 · HTML: https://mlsanity.com/paper/2607.05936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG for code, one of 15 directions in Retrieval-augmented generation.
- Builds on 16 of its 59 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Placed in

- Retrieval-augmented generation / [RAG for code](https://mlsanity.com/rag/rag-for-code/)

## Builds on (most cited first, 16 of 16)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Survey on Evaluation of LLM-based Agents](https://mlsanity.com/paper/2503.16416/) (2025)
- [Efficient Guided Generation for Large Language Models](https://mlsanity.com/paper/2307.09702/) (2023)
- [RestGPT: Connecting Large Language Models with Real-World RESTful APIs](https://mlsanity.com/paper/2306.06624/) (2023)
- [What's Wrong with Your Code Generated by Large Language Models? An Extensive Study](https://mlsanity.com/paper/2407.06153/) (2024)
- [RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.03275/) (2025)
- [De-Hallucinator: Mitigating LLM Hallucinations in Code Generation Tasks via Iterative Grounding](https://mlsanity.com/paper/2401.01701/) (2024)
- [Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decoding](https://mlsanity.com/paper/2310.07075/) (2023)
- [Applying RLAIF for Code Generation with API-usage in Lightweight LLMs](https://mlsanity.com/paper/2406.20060/) (2024)
- [Dialog2API: Task-Oriented Dialogue with API Description and Example Programs](https://mlsanity.com/paper/2212.09946/) (2022)
- [Compositional Generalization for Natural Language Interfaces to Web APIs](https://mlsanity.com/paper/2112.05209/) (2021)
- [Pop Quiz! Do Pre-trained Code Models Possess Knowledge of Correct API Names?](https://mlsanity.com/paper/2309.07804/) (2023)

## Abstract (arXiv)

Integration of web APIs is a cornerstone of modern software systems, yet writing correct web API invocation code remains challenging due to complex and evolving API specifications. Although LLMs are increasingly used for code generation, previous work has empirically shown that their ability to generate correct web API integrations is limited. At the same time, mitigation techniques and their effectiveness for this setting remain insufficiently understood. In this paper, we propose and systematically evaluate retrieval-augmented generation (RAG) and constrained decoding (CD) as two complementary approaches to improving LLM-generated web API invocation code. For RAG, we design a retriever that processes OpenAPI specifications and retrieves compact endpoint representations to inject into model prompts. For CD, we introduce an automatic translation from OpenAPI specifications to regex-based constraints enforced during generation. We evaluate both approaches on WAPIIBench's existing synthetic dataset and on a new real-world dataset derived from GitHub repositories. Our results show that RAG reduces hallucinations and improves correctness when generating full API invocations but reduces it when the endpoint is already provided as it encourages the generation of unnecessary parameters. In contrast, CD reliably prevents illegal URLs, HTTP methods, and arguments and substantially improves overall correctness for both starter codes.