# Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation

arXiv: https://arxiv.org/abs/2407.19619 · HTML: https://mlsanity.com/paper/2407.19619/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in RAG for code, one of 15 directions in Retrieval-augmented generation.
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Placed in

- Retrieval-augmented generation / [RAG for code](https://mlsanity.com/rag/rag-for-code/)

## Builds on (most cited first, 15 of 15)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [Granite Code Models: A Family of Open Foundation Models for Code Intelligence](https://mlsanity.com/paper/2405.04324/) (2024)
- [Fine-tuning Large Language Models for Domain-specific Machine Translation](https://mlsanity.com/paper/2402.15061/) (2024)
- [Creating a Dataset for High-Performance Computing Code Translation using LLMs: A Bridge Between OpenMP Fortran and C++](https://mlsanity.com/paper/2307.07686/) (2023)

## Built on by (most cited first, 16 of 16)

- [RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation](https://mlsanity.com/paper/2412.17744/) (2024)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval](https://mlsanity.com/paper/2505.05666/) (2025)
- [LLM-Assisted Translation of Legacy FORTRAN Codes to C++: A Cross-Platform Study](https://mlsanity.com/paper/2504.15424/) (2025)
- [ARCS: Agentic Retrieval-Augmented Code Synthesis with Iterative Refinement](https://mlsanity.com/paper/2504.20434/) (2025)
- [Function-to-Style Guidance of LLMs for Code Translation](https://mlsanity.com/paper/2507.11083/) (2025)
- [Skeleton-Guided-Translation: A Benchmarking Framework for Code Repository Translation with Fine-Grained Quality Evaluation](https://mlsanity.com/paper/2501.16050/) (2025)
- [Advances in LLMs with Focus on Reasoning, Adaptability, Efficiency and Ethics](https://mlsanity.com/paper/2506.12365/) (2025)
- [Benchmarking Large Language Models with Integer Sequence Generation Tasks](https://mlsanity.com/paper/2411.04372/) (2024)
- [Optimizing Code Runtime Performance through Context-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2501.16692/) (2025)
- [Performance Evaluation of General Purpose Large Language Models for Basic Linear Algebra Subprograms Code Generation](https://mlsanity.com/paper/2507.04697/) (2025)
- [BabelCoder: Agentic Code Translation with Specification Alignment](https://mlsanity.com/paper/2512.06902/) (2025)
- [CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora](https://mlsanity.com/paper/2604.18027/) (2026)
- [Context-Aware Search and Retrieval Over Erasure Channels](https://mlsanity.com/paper/2507.11894/) (2025)
- [Bridging Functional Correctness and Runtime Efficiency Gaps in LLM-Based Code Translation](https://mlsanity.com/paper/2606.17683/) (2026)
- [ORCA: Evaluating LLMs on Data Science Code Translation](https://mlsanity.com/paper/2609.30749/) (2026)

## Abstract (arXiv)

The advent of large language models (LLMs) has significantly advanced the field of code translation, enabling automated translation between programming languages. However, these models often struggle with complex translation tasks due to inadequate contextual understanding. This paper introduces a novel approach that enhances code translation through Few-Shot Learning, augmented with retrieval-based techniques. By leveraging a repository of existing code translations, we dynamically retrieve the most relevant examples to guide the model in translating new code segments. Our method, based on Retrieval-Augmented Generation (RAG), substantially improves translation quality by providing contextual examples from which the model can learn in real-time. We selected RAG over traditional fine-tuning methods due to its ability to utilize existing codebases or a locally stored corpus of code, which allows for dynamic adaptation to diverse translation tasks without extensive retraining. Extensive experiments on diverse datasets with open LLM models such as Starcoder, Llama3-70B Instruct, CodeLlama-34B Instruct, Granite-34B Code Instruct, and Mixtral-8x22B, as well as commercial LLM models like GPT-3.5 Turbo and GPT-4o, demonstrate our approach's superiority over traditional zero-shot methods, especially in translating between Fortran and CPP. We also explored varying numbers of shots i.e. examples provided during inference, specifically 1, 2, and 3 shots and different embedding models for RAG, including Nomic-Embed, Starencoder, and CodeBERT, to assess the robustness and effectiveness of our approach.