mlsanityData to 7 Oct 2026

FRANC: A Lightweight Framework for High-Quality Code Generation

arXiv 2307.08220 ↗First posted 17 Jul 2023cs.SE cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

In recent years, the use of automated source code generation utilizing transformer-based generative models has expanded, and these models can generate functional code according to the requirements of the developers. However, recent research revealed that these automatically generated source codes can contain vulnerabilities and other quality issues. Despite researchers' and practitioners' attempts to enhance code generation models, retraining and fine-tuning large language models is time-consuming and resource-intensive. Thus, we describe FRANC, a lightweight framework for recommending more secure and high-quality source code derived from transformer-based code generation models. FRANC includes a static filter to make the generated code compilable with heuristics and a quality-aware ranker to sort the code snippets based on a quality score. Moreover, the framework uses prompt engineering to fix persistent quality issues. We evaluated the framework with five Python and Java code generation models and six prompt datasets, including a newly created one in this work (SOEval). The static filter improves 9% to 46% Java suggestions and 10% to 43% Python suggestions regarding compilability. The average improvement over the NDCG@10 score for the ranking system is 0.0763, and the repairing techniques repair the highest 80% of prompts. FRANC takes, on average, 1.98 seconds for Java; for Python, it takes 0.08 seconds.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 38 of its 78 references that are in the map. First posted on arXiv 17 Jul 2023.

Builds on

38 of 78 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  10. 2023Textbooks Are All You Needcited by 561
  11. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  12. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
26 more
  1. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
  2. 2022CodeT: Code Generation with Generated Testscited by 369
  3. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  4. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  5. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  6. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  7. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  8. 2022Multi-lingual Evaluation of Code Generation Modelscited by 158
  9. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
  10. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  11. 2021Examining Zero-Shot Vulnerability Repair with Large Language Modelscited by 105
  12. 2022Productivity Assessment of Neural Code Completioncited by 97
  13. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 75
  14. 2020Code Prediction by Feeding Trees to Transformerscited by 70
  15. 2022Repair Is Nearly Generation: Multilingual Program Repair with LLMscited by 68
  16. 2023CodeScore: Evaluating Code Generation by Learning Code Executioncited by 66
  17. 2020Deep Learning for Source Code Modeling and Generation: Models, Applications and Challengescited by 55
  18. 2023Using Large Language Models to Generate JUnit Tests: An Empirical Studycited by 53
  19. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 49
  20. 2023SkCoder: A Sketch-based Approach for Automatic Code Generationcited by 40
  21. 2020Fast and Memory-Efficient Neural Code Completioncited by 38
  22. 2022Fault-Aware Neural Code Rankerscited by 33
  23. 2022AixBench: A Code Generation Benchmark Datasetcited by 22
  24. 2022M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarizationcited by 14
  25. 2023Towards Generating Functionally Correct Code Edits from Natural Language Issue Descriptionscited by 14
  26. 2020Security Smells in Androidcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 3; 2026: 2.

14 papers.

Sort:

By area:

  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  2. 2024CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferencescited by 56
  3. 2023SALLM: Security Assessment of Generated Codecited by 35
  4. 2024Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agentscited by 25
  5. 2024LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systemscited by 20
  6. 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHubcited by 6
  7. 2025Standardizing Intelligence: Aligning Generative AI for Regulatory and Operational Compliancecited by 5
  8. 2024Conversational Code Generation: a Case Study of Designing a Dialogue System for Generating Driving Scenarios for Testing Autonomous Vehiclescited by 4
  9. 2023Can LLMs Patch Security Issues?cited by 3
  10. 2025Prompt engineering and framework: implementation to increase code reliability based guideline for LLMscited by 1
  11. 2024Automated Creation of Source Code Variants of a Cryptographic Hash Function Implementation Using Generative Pre-Trained Transformer Modelscited by 0
  12. 2024Demo: SGCode: A Flexible Prompt-Optimizing System for Secure Generation of Codecited by 0
2 more
  1. 2025Assessing the Software Security Comprehension of Large Language Modelscited by 0
  2. 2026Inference-Time Safety For Code LLMs Via Retrieval-Augmented Revisioncited by 0