Bridging LLM-Generated Code and Requirements: Reverse Generation technique and SBC Metric for Developer Insights
Not on the map. Why?
Abstract (arXiv)
The rise of Large Language Models (LLMs) in software engineering, particularly in code generation, has garnered significant attention. However, assessing the quality of AI-generated code remains a challenge due to the inherent complexity of programming tasks and the lack of robust evaluation metrics that align well with human judgment. Traditional token-based metrics such as BLEU and ROUGE, while commonly used in natural language processing, exhibit weak correlations with human assessments in code intelligence and verification tasks. Furthermore, these metrics are primarily research focused and are not designed for seamless integration into the software development lifecycle, limiting their practical utility for developers seeking to improve code quality and security. AI-assisted coding has been shown to be more beneficial for senior developers, as they possess the expertise to critically evaluate the generated code for correctness, completeness, and compliance. In contrast, junior developers may struggle to identify hallucinations, missing functionality, or incorrect logic in AI-generated code. To bridge this gap, This paper introduces a novel scoring mechanism called the SBC score, which is based on a reverse generation technique that leverages the natural language generation capabilities of LLMs. Unlike direct code analysis, our approach reconstructs system requirements from AI-generated code and compares them with the original specifications to quantify accuracy. The SBC score combines semantic similarity, BLEU, and completeness analysis, providing actionable insights to developers by highlighting missing features and hallucinations. Our code and datasets are available on GitHub
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 11 Feb 2025.
Builds on
12 of 18 references.
References · 12
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
- 2019SPoC: Search-based Pseudocode to Codecited by 176
- 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
- 2023CodeBERTScore: Evaluating Code Generation with Pretrained Models of Codecited by 102
- 2022Out of the BLEU: how should we assess quality of the Code Generation models?cited by 65
- 2023ICE-Score: Instructing Large Language Models to Evaluate Codecited by 52
- 2024CodeJudge: Evaluating Code Generation with Large Language Modelscited by 44
- 2024The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligencecited by 23
- 2024Validating LLM-Generated Programs with Metamorphic Prompt Testingcited by 6
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
- 2025Application Modernization with LLMs: Addressing Core Challenges in Reliability, Security, and Qualitycited by 0
- 2026EstRTL: Functional Estimation Guided RTL Code Generationcited by 0
- 2026EstRTL: Functional Estimation Guided RTL Code Generation1 Jun 2026
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence23 Nov 2025
- 2025Application Modernization with LLMs: Addressing Core Challenges in Reliability, Security, and Quality13 Feb 2025