mlsanityData to 14 Oct 2026

IaC Generation with LLMs: An Error Taxonomy and A Study on Configuration Knowledge Injection

arXiv 2512.14792 ↗First posted 16 Dec 2025cs.AI cs.SEcited by 2

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

Large Language Models (LLMs) currently exhibit low success rates in generating correct and intent-aligned Infrastructure as Code (IaC). This research investigated methods to improve LLM-based IaC generation, specifically for Terraform, by systematically injecting structured configuration knowledge. To facilitate this, an existing IaC-Eval benchmark was significantly enhanced with cloud emulation and automated error analysis. Additionally, a novel error taxonomy for LLM-assisted IaC code generation was developed. A series of knowledge injection techniques was implemented and evaluated, progressing from Naive Retrieval-Augmented Generation (RAG) to more sophisticated Graph RAG approaches. These included semantic enrichment of graph components and modeling inter-resource dependencies. Experimental results demonstrated that while baseline LLM performance was poor (27.1% overall success), injecting structured configuration knowledge increased technical validation success to 75.3% and overall success to 62.6%. Despite these gains in technical correctness, intent alignment plateaued, revealing a "Correctness-Congruence Gap" where LLMs can become proficient "coders" but remain limited "architects" in fulfilling nuanced user intent.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 33 of its 71 references that are in the map. First posted on arXiv 16 Dec 2025.

Builds on

33 of 71 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  9. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  11. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  12. 2020Knowledge Graphscited by 621
21 more
  1. 2018Deep learning for time series classification: a reviewcited by 576
  2. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  3. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  4. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  5. 2022Large Language Models Meet NL2Code: A Surveycited by 138
  6. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 127
  7. 2023Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issuescited by 79
  8. 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 73
  9. 2024Bugs in Large Language Models Generated Code: An Empirical Studycited by 60
  10. 2025Knowledge Graph-Guided Retrieval Augmented Generationcited by 58
  11. 2023Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancementcited by 14
  12. 2024Advanced RAG Models with Graph Structures: Optimizing Complex Knowledge Reasoning and Text Generationcited by 13
  13. 2019A Survey of DevOps Concepts and Challengescited by 12
  14. 2024Insights into Natural Language Database Query Errors: From Attention Misalignment to User Handling Strategiescited by 12
  15. 2024DocCGen: Document-based Controlled Code Generationcited by 11
  16. 2024A Survey of using Large Language Models for Generating Infrastructure as Codecited by 10
  17. 2025Cloud Infrastructure Management in the Age of AI Agentscited by 9
  18. 2023CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generationcited by 8
  19. 2024Large Language Model Enhanced Knowledge Representation Learning: A Surveycited by 7
  20. 2021KGSecConfig: A Knowledge Graph Based Approach for Secured Container Orchestrator Configurationcited by 5
  21. 2020The 'as Code' Activities: Development Anti-patterns for Infrastructure as Codecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisitioncited by 0
  2. 2026Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Verifier-First Evaluation of Agentic LLMs for Infrastructure-as-Code Generation
  2. 2026TerraFormer: Automated Infrastructure-as-Code with LLMs Fine-Tuned via Policy-Guided Verifier Feedback
  3. 2024Using a Feedback Loop for LLM-based Infrastructure as Code Generation
  4. 2026TerraRepair: A Tool-Grounded LLM Agent for Infrastructure-as-Code Repair
  5. 2025Multi-Agent Code-Orchestrated Generation for Reliable Infrastructure-as-Code
  6. 2025Towards Advancing Code Generation with Large Language Models: A Research Roadmap
  7. 2025Holistic Evaluation of State-of-the-Art LLMs for Code Generation
  8. 2026SWE-InfraBench: Evaluating Language Models on Cloud Infrastructure Code ↗