# Variable Name Recovery in Decompiled Binary Code using Constrained Masked Language Modeling

arXiv: https://arxiv.org/abs/2103.12801 · HTML: https://mlsanity.com/paper/2103.12801/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [pix2code: Generating Code from a Graphical User Interface Screenshot](https://mlsanity.com/paper/1705.07962/) (2017)
- [Abstract Syntax Networks for Code Generation and Semantic Parsing](https://mlsanity.com/paper/1704.07535/) (2017)
- [Latent Predictor Networks for Code Generation](https://mlsanity.com/paper/1603.06744/) (2016)
- [DIRE: A Neural Approach to Decompiled Identifier Naming](https://mlsanity.com/paper/1909.09029/) (2019)
- [On the "Naturalness" of Buggy Code](https://mlsanity.com/paper/1506.01159/) (2015)
- [Learning Natural Coding Conventions](https://mlsanity.com/paper/1402.4182/) (2014)
- [Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks](https://mlsanity.com/paper/1603.06129/) (2016)
- [Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts](https://mlsanity.com/paper/1809.05193/) (2018)
- [sk_p: a neural program corrector for MOOCs](https://mlsanity.com/paper/1607.02902/) (2016)
- [Deep Reinforcement Learning for Programming Language Correction](https://mlsanity.com/paper/1801.10467/) (2018)
- [In Nomine Function: Naming Functions in Stripped Binaries with Neural Networks](https://mlsanity.com/paper/1912.07946/) (2019)
- [CodeSum: Translate Program Language to Natural Language](https://mlsanity.com/paper/1708.01837/) (2017)
- [Neural Reverse Engineering of Stripped Binaries using Augmented Control Flow Graphs](https://mlsanity.com/paper/1902.09122/) (2019)

## Built on by (most cited first, 7 of 7)

- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries](https://mlsanity.com/paper/2301.01701/) (2023)
- [Binary Code Summarization: Benchmarking ChatGPT/GPT-4 and Other Large Language Models](https://mlsanity.com/paper/2312.09601/) (2023)
- [Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases](https://mlsanity.com/paper/2405.19581/) (2024)
- [FidelityGPT: Correcting Decompilation Distortions with Retrieval Augmented Generation](https://mlsanity.com/paper/2510.19615/) (2025)
- [Unmasking the Shadows: Pinpoint the Implementations of Anti-Dynamic Analysis Techniques in Malware Using LLM](https://mlsanity.com/paper/2411.05982/) (2024)
- [REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)](https://mlsanity.com/paper/2604.27319/) (2026)

## Abstract (arXiv)

Decompilation is the procedure of transforming binary programs into a high-level representation, such as source code, for human analysts to examine. While modern decompilers can reconstruct and recover much information that is discarded during compilation, inferring variable names is still extremely difficult. Inspired by recent advances in natural language processing, we propose a novel solution to infer variable names in decompiled code based on Masked Language Modeling, Byte-Pair Encoding, and neural architectures such as Transformers and BERT. Our solution takes textitraw decompiler output, the less semantically meaningful code, as input, and enriches it using our proposed textitfinetuning technique, Constrained Masked Language Modeling. Using Constrained Masked Language Modeling introduces the challenge of predicting the number of masked tokens for the original variable name. We address this textitcount of token prediction challenge with our post-processing algorithm. Compared to the state-of-the-art approaches, our trained VarBERT model is simpler and of much better performance. We evaluated our model on an existing large-scale data set with 164,632 binaries and showed that it can predict variable names identical to the ones present in the original source code up to 84.15% of the time.