mlsanityData to 14 Oct 2026

Vibe-Coding: Feedback-Based Automated Verification with no Human Code Inspection, a Feasibility Study

arXiv 2604.14867 ↗First posted 16 Apr 2026cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Vibe coding inherently assumes iterative refinement of LLM-generated code through feedback loops. While effective for conventional software tasks, its reliability in runtime-adaptive systems is unclear -- especially when generated code is not manually inspected. This paper studies feedback-based automated verification of LLM-generated adaptation managers in Collective Adaptive Systems (CAS). We focus on the key challenges of verification in the loop: how to detect failures of generated code at runtime and how to report them precisely enough for an LLM to fix them. We combine the adaptation loop with a vibe-coding feedback loop where correctness is checked against (i) generic architectural constraints and (ii) functional constraints formalized in Functional Constraints Logic (FCL), a novel first-order temporal logic over potentially finite traces. Conducting the Dragon Hunt CAS case study, we show that fine-grained constraint violations provide actionable feedback that typically yields a valid adaptation manager within a few iterations, while simple coarse metric-based feedback often stalls. Our findings suggest that feedback precision is the dominant factor for reliable vibe coding in systems designed by domain experts with no programming skills, thereby obviating the need for human code inspection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 14 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

6 of 14 references.

References · 6

  1. 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agentscited by 133
  2. 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 73
  3. 2024AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testingcited by 35
  4. 2025RvLLM: LLM Runtime Verification with Domain Knowledgecited by 17
  5. 2025Towards Formal Verification of LLM-Generated Code from Natural Language Promptscited by 8
  6. 2026Feedback-based Automated Verification in Vibe Coding of CAS Adaptation Built on Constraint Logiccited by 1

Built on by

Not cited yet.