mlsanityData to 7 Oct 2026

Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation

arXiv 2512.00134 ↗First posted 28 Nov 2025cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Assembly-to-source code translation is a critical task in reverse engineering, cybersecurity, and software maintenance, yet systematic benchmarks for evaluating large language models on this problem remain scarce. In this work, we present the first comprehensive evaluation of five state-of-the-art large language models on assembly-to-source translation. We assess model performance using a diverse set of metrics capturing lexical similarity (BLEU, ROUGE, and METEOR), semantic alignment (BERTScore), fluency (Perplexity), and efficiency (time prediction). Our results reveal clear trade-offs: while certain models excel in text similarity metrics, others demonstrate lower perplexity or faster inference times. We further provide qualitative analyses of typical model successes and failure cases, highlighting challenges such as control flow recovery and identifier reconstruction. Taken together, our benchmark offers actionable insights into the strengths and limitations of current large language models for program translation, establishing a foundation for future research in combining accuracy with efficiency for real-world applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 28 Nov 2025.

Builds on

25 of 40 references.

References · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  6. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  7. 2024Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Modelscited by 83
  8. 2021Language Model Evaluation Beyond Perplexitycited by 77
  9. 2019Does BLEU Score Work for Code Migration?cited by 49
  10. 2022BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generationcited by 48
  11. 2024LLM4Decompile: Decompiling Binary Code with Large Language Modelscited by 38
  12. 2019DIRE: A Neural Approach to Decompiled Identifier Namingcited by 36
13 more
  1. 2023Refining Decompiled C Code with Large Language Modelscited by 24
  2. 2021Augmenting Decompiler Output with Learned Variable Names and Typescited by 23
  3. 2024Unveiling LLM Evaluation Focused on Metrics: Challenges and Solutionscited by 17
  4. 2022Beyond the C: Retargetable Decompilation using Neural Machine Translationcited by 15
  5. 2025Large Language Models (LLMs) for Source Code Analysis: applications, models and datasetscited by 15
  6. 2025Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineeringcited by 15
  7. 2023Boosting Neural Networks to Decompile Optimized Binariescited by 11
  8. 2025Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilationcited by 11
  9. 2024Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMscited by 9
  10. 2025SBAN: A Framework & Multi-Dimensional Dataset for Large Language Model Pre-Training and Software Code Miningcited by 3
  11. 2025Decompiling Rust: An Empirical Study of Compiler Optimizations and Reverse Engineering Challengescited by 1
  12. 2025NLD-LLM: A systematic framework for evaluating small language transformer models on natural language descriptioncited by 1
  13. 2025Can Small GenAI Language Models Rival Large Language Models in Understanding Application Behavior?cited by 1

Built on by

Not cited yet.