# NeuDep: Neural Binary Memory Dependence Analysis

arXiv: https://arxiv.org/abs/2210.02853 · HTML: https://mlsanity.com/paper/2210.02853/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 108 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Leveraging Automated Unit Tests for Unsupervised Code Translation](https://mlsanity.com/paper/2110.06773/) (2021)
- [A Syntax-Guided Edit Decoder for Neural Program Repair](https://mlsanity.com/paper/2106.08253/) (2021)
- [CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences](https://mlsanity.com/paper/2202.06689/) (2022)
- [Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations](https://mlsanity.com/paper/2009.02731/) (2020)
- [PalmTree: Learning an Assembly Language Model for Instruction Embedding](https://mlsanity.com/paper/2103.03809/) (2021)
- [On the Evaluation of Neural Code Summarization](https://mlsanity.com/paper/2107.07112/) (2021)
- [Trex: Learning Execution Semantics from Micro-Traces for Binary Similarity](https://mlsanity.com/paper/2012.08680/) (2020)
- [TypeWriter: Neural Type Prediction with Search-based Validation](https://mlsanity.com/paper/1912.03768/) (2019)
- [Learning Natural Coding Conventions](https://mlsanity.com/paper/1402.4182/) (2014)
- [A Grammar-Based Structural CNN Decoder for Code Generation](https://mlsanity.com/paper/1811.06837/) (2018)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Dynamic Neural Program Embedding for Program Repair](https://mlsanity.com/paper/1711.07163/) (2017)
- [InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees](https://mlsanity.com/paper/2012.07023/) (2020)
- [An Empirical Study on the Usage of BERT Models for Code Completion](https://mlsanity.com/paper/2103.07115/) (2021)
- [XDA: Accurate, Robust Disassembly with Transfer Learning](https://mlsanity.com/paper/2010.00770/) (2020)
- [Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations](https://mlsanity.com/paper/2111.10793/) (2021)
- [Review4Repair: Code Review Aided Automatic Program Repairing](https://mlsanity.com/paper/2010.01544/) (2020)
- [Cross-Language Binary-Source Code Matching with Intermediate Representations](https://mlsanity.com/paper/2201.07420/) (2022)
- [Learning Python Code Suggestion with a Sparse Pointer Network](https://mlsanity.com/paper/1611.08307/) (2016)
- [CRaDLe: Deep Code Retrieval Based on Semantic Dependency Learning](https://mlsanity.com/paper/2012.01028/) (2020)
- [Neural Software Analysis](https://mlsanity.com/paper/2011.07986/) (2020)
- [GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses](https://mlsanity.com/paper/2112.01218/) (2021)
- [Modeling Functional Similarity in Source Code with Graph-Based Siamese Networks](https://mlsanity.com/paper/2011.11228/) (2020)
- [Deep Learning & Software Engineering: State of Research and Future Directions](https://mlsanity.com/paper/2009.08525/) (2020)
- [Nalin: Learning from Runtime Behavior to Find Name-Value Inconsistencies in Jupyter Notebooks](https://mlsanity.com/paper/2112.06186/) (2021)

## Built on by (most cited first, 3 of 3)

- [Binary Code Summarization: Benchmarking ChatGPT/GPT-4 and Other Large Language Models](https://mlsanity.com/paper/2312.09601/) (2023)
- [A Progressive Transformer for Unifying Binary Code Embedding and Knowledge Transfer](https://mlsanity.com/paper/2412.11177/) (2024)
- [SoK: AI-Augmented Binary Reversing](https://mlsanity.com/paper/2606.17398/) (2026)

## Abstract (arXiv)

Determining whether multiple instructions can access the same memory location is a critical task in binary analysis. It is challenging as statically computing precise alias information is undecidable in theory. The problem aggravates at the binary level due to the presence of compiler optimizations and the absence of symbols and types. Existing approaches either produce significant spurious dependencies due to conservative analysis or scale poorly to complex binaries. We present a new machine-learning-based approach to predict memory dependencies by exploiting the model's learned knowledge about how binary programs execute. Our approach features (i) a self-supervised procedure that pretrains a neural net to reason over binary code and its dynamic value flows through memory addresses, followed by (ii) supervised finetuning to infer the memory dependencies statically. To facilitate efficient learning, we develop dedicated neural architectures to encode the heterogeneous inputs (i.e., code, data values, and memory addresses from traces) with specific modules and fuse them with a composition learning strategy. We implement our approach in NeuDep and evaluate it on 41 popular software projects compiled by 2 compilers, 4 optimizations, and 4 obfuscation passes. We demonstrate that NeuDep is more precise (1.5x) and faster (3.5x) than the current state-of-the-art. Extensive probing studies on security-critical reverse engineering tasks suggest that NeuDep understands memory access patterns, learns function signatures, and is able to match indirect calls. All these tasks either assist or benefit from inferring memory dependencies. Notably, NeuDep also outperforms the current state-of-the-art on these tasks.