mlsanityData to 14 Oct 2026

Novel Preprocessing Technique for Data Embedding in Engineering Code Generation Using Large Language Model

arXiv 2311.16267 ↗First posted 27 Nov 2023cs.CL cs.SEcited by 2

Not on the map. Why?

Abstract (arXiv)

We present four main contributions to enhance the performance of Large Language Models (LLMs) in generating domain-specific code: (i) utilizing LLM-based data splitting and data renovation techniques to improve the semantic representation of embeddings' space; (ii) introducing the Chain of Density for Renovation Credibility (CoDRC), driven by LLMs, and the Adaptive Text Renovation (ATR) algorithm for assessing data renovation reliability; (iii) developing the Implicit Knowledge Expansion and Contemplation (IKEC) Prompt technique; and (iv) effectively refactoring existing scripts to generate new and high-quality scripts with LLMs. By using engineering simulation software RedHawk-SC as a case study, we demonstrate the effectiveness of our data pre-processing method for expanding and categorizing scripts. When combined with IKEC, these techniques enhance the Retrieval-Augmented Generation (RAG) method in retrieving more relevant information, ultimately achieving a 73.33% "Percentage of Correct Lines" for code generation problems in MapReduce applications.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

25 of 33 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  12. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
13 more
  1. 2023Large Language Models as Optimizerscited by 864
  2. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  3. 2023Instruction Tuning with GPT-4cited by 686
  4. 2023Active Retrieval Augmented Generationcited by 516
  5. 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 200
  6. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 194
  7. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 159
  8. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  9. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 146
  10. 2023Self-planning Code Generation with Large Language Modelscited by 124
  11. 2023ChatEDA: A Large Language Model Powered Autonomous Agent for EDAcited by 122
  12. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  13. 2023From Sparse to Dense: GPT-4 Summarization with Chain of Density Promptingcited by 52

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Retrieval-Augmented Test Generation: How Far Are We?cited by 7
  2. 2025JARVIS: A Multi-Agent Code Assistant for High-Quality EDA Script Generationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards A Generalist Code Embedding Model Based On Massive Data Synthesis
  2. 2024Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning
  3. 2024Improving the Ability of Pre-trained Language Model by Imparting Large Language Model's Experience ↗
  4. 2026MAP4CS: A Multi-dimensional Data Pruning Framework for Efficient Code Retriever Fine-tuning
  5. 2025FastCoder: Accelerating Repository-level Code Generation via Efficient Retrieval and Verification
  6. 2026Context-Augmented Code Generation Using Programming Knowledge Graphs
  7. 2026Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
  8. 2023SelfEvolve: A Code Evolution Framework via Large Language Models