# Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation

arXiv: https://arxiv.org/abs/2512.00134 · HTML: https://mlsanity.com/paper/2512.00134/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models](https://mlsanity.com/paper/2405.20541/) (2024)
- [Language Model Evaluation Beyond Perplexity](https://mlsanity.com/paper/2106.00085/) (2021)
- [Does BLEU Score Work for Code Migration?](https://mlsanity.com/paper/1906.04903/) (2019)
- [BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation](https://mlsanity.com/paper/2210.07626/) (2022)
- [LLM4Decompile: Decompiling Binary Code with Large Language Models](https://mlsanity.com/paper/2403.05286/) (2024)
- [DIRE: A Neural Approach to Decompiled Identifier Naming](https://mlsanity.com/paper/1909.09029/) (2019)
- [Refining Decompiled C Code with Large Language Models](https://mlsanity.com/paper/2310.06530/) (2023)
- [Augmenting Decompiler Output with Learned Variable Names and Types](https://mlsanity.com/paper/2108.06363/) (2021)
- [Unveiling LLM Evaluation Focused on Metrics: Challenges and Solutions](https://mlsanity.com/paper/2404.09135/) (2024)
- [Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets](https://mlsanity.com/paper/2503.17502/) (2025)
- [Beyond the C: Retargetable Decompilation using Neural Machine Translation](https://mlsanity.com/paper/2212.08950/) (2022)
- [Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineering](https://mlsanity.com/paper/2504.07137/) (2025)
- [Boosting Neural Networks to Decompile Optimized Binaries](https://mlsanity.com/paper/2301.00969/) (2023)
- [Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilation](https://mlsanity.com/paper/2505.12668/) (2025)
- [Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMs](https://mlsanity.com/paper/2406.01943/) (2024)
- [SBAN: A Framework & Multi-Dimensional Dataset for Large Language Model Pre-Training and Software Code Mining](https://mlsanity.com/paper/2510.18936/) (2025)
- [Decompiling Rust: An Empirical Study of Compiler Optimizations and Reverse Engineering Challenges](https://mlsanity.com/paper/2507.18792/) (2025)
- [NLD-LLM: A systematic framework for evaluating small language transformer models on natural language description](https://mlsanity.com/paper/2510.05139/) (2025)
- [Can Small GenAI Language Models Rival Large Language Models in Understanding Application Behavior?](https://mlsanity.com/paper/2511.12576/) (2025)

## Abstract (arXiv)

Assembly-to-source code translation is a critical task in reverse engineering, cybersecurity, and software maintenance, yet systematic benchmarks for evaluating large language models on this problem remain scarce. In this work, we present the first comprehensive evaluation of five state-of-the-art large language models on assembly-to-source translation. We assess model performance using a diverse set of metrics capturing lexical similarity (BLEU, ROUGE, and METEOR), semantic alignment (BERTScore), fluency (Perplexity), and efficiency (time prediction). Our results reveal clear trade-offs: while certain models excel in text similarity metrics, others demonstrate lower perplexity or faster inference times. We further provide qualitative analyses of typical model successes and failure cases, highlighting challenges such as control flow recovery and identifier reconstruction. Taken together, our benchmark offers actionable insights into the strengths and limitations of current large language models for program translation, establishing a foundation for future research in combining accuracy with efficiency for real-world applications.