mlsanityData to 14 Oct 2026

Automated Creation of Source Code Variants of a Cryptographic Hash Function Implementation Using Generative Pre-Trained Transformer Models

arXiv 2404.15681 ↗First posted 24 Apr 2024cs.CR cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative pre-trained transformers (GPT's) are a type of large language machine learning model that are unusually adept at producing novel, and coherent, natural language. In this study the ability of GPT models to generate novel and correct versions, and notably very insecure versions, of implementations of the cryptographic hash function SHA-1 is examined. The GPT models Llama-2-70b-chat-h, Mistral-7B-Instruct-v0.1, and zephyr-7b-alpha are used. The GPT models are prompted to re-write each function using a modified version of the localGPT framework and langchain to provide word embedding context of the full source code and header files to the model, resulting in over 150,000 function re-write GPT output text blocks, approximately 50,000 of which were able to be parsed as C code and subsequently compiled. The generated code is analyzed for being compilable, correctness of the algorithm, memory leaks, compiler optimization stability, and character distance to the reference implementation. Remarkably, several generated function variants have a high implementation security risk of being correct for some test vectors, but incorrect for other test vectors. Additionally, many function implementations were not correct to the reference algorithm of SHA-1, but produced hashes that have some of the basic characteristics of hash functions. Many of the function re-writes contained serious flaws such as memory leaks, integer overflows, out of bounds accesses, use of uninitialised values, and compiler optimization instability. Compiler optimization settings and SHA-256 hash checksums of the compiled binaries are used to cluster implementations that are equivalent but may not have identical syntax - using this clustering over 100,000 novel and correct versions of the SHA-1 codebase were generated where each component C function of the reference implementation is different from the original code.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 94 references that are in the map. First posted on arXiv 24 Apr 2024.

Builds on

33 of 94 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2023Large Language Models as Optimizerscited by 864
  10. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  11. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  12. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
21 more
  1. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  2. 2020IntelliCode Compose: Code Generation Using Transformercited by 227
  3. 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 200
  4. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  5. 2020Unit Test Case Generation with Transformers and Focal Contextcited by 118
  6. 2023InferFix: End-to-End Program Repair with LLMscited by 114
  7. 2021Examining Zero-Shot Vulnerability Repair with Large Language Modelscited by 105
  8. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 91
  9. 2024Calibration and Correctness of Language Models for Codecited by 49
  10. 2023Large Language Model-Aware In-Context Learning for Code Generationcited by 43
  11. 2023DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructionscited by 33
  12. 2023ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generationcited by 32
  13. 2023Comparing Llama-2 and GPT-3 LLMs for HPC kernels generationcited by 30
  14. 2023FRANC: A Lightweight Framework for High-Quality Code Generationcited by 14
  15. 2024Copilot Evaluation Harness: Evaluating LLM-Guided Software Programmingcited by 13
  16. 2021Automatic Code Generation using Pre-Trained Language Modelscited by 8
  17. 2023Fixing Rust Compilation Errors using LLMscited by 7
  18. 2021CGEMs: A Metric Model for Automatic Code Generation using GPT-3cited by 6
  19. 2023Prompting Code Interpreter to Write Better Unit Tests on Quixbugs Functionscited by 3
  20. 2022Use of Cryptography in Malware Obfuscationcited by 2
  21. 2023Cryptographic ransomware encryption detection: Surveycited by 2

Built on by

Not cited yet.