# Multi-modal Learning for WebAssembly Reverse Engineering

arXiv: https://arxiv.org/abs/2404.03171 · HTML: https://mlsanity.com/paper/2404.03171/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 72 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [Improving Automatic Source Code Summarization via Deep Reinforcement Learning](https://mlsanity.com/paper/1811.07234/) (2018)
- [A Neural Model for Generating Natural Language Summaries of Program Subroutines](https://mlsanity.com/paper/1902.01954/) (2019)
- [PalmTree: Learning an Assembly Language Model for Instruction Embedding](https://mlsanity.com/paper/2103.03809/) (2021)
- [jTrans: Jump-Aware Transformer for Binary Code Similarity](https://mlsanity.com/paper/2205.12713/) (2022)
- [Deep API Learning](https://mlsanity.com/paper/1605.08535/) (2016)
- [Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code](https://mlsanity.com/paper/2205.11739/) (2022)
- [A Literature Study of Embeddings on Source Code](https://mlsanity.com/paper/1904.03061/) (2019)
- [A deep language model for software code](https://mlsanity.com/paper/1608.02715/) (2016)
- [Neural Software Analysis](https://mlsanity.com/paper/2011.07986/) (2020)
- [Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics](https://mlsanity.com/paper/1801.05613/) (2018)
- [Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection](https://mlsanity.com/paper/2108.06082/) (2021)
- [Public Git Archive: a Big Code dataset for all](https://mlsanity.com/paper/1803.10144/) (2018)
- [CT-Wasm: Type-Driven Secure Cryptography for the Web Ecosystem](https://mlsanity.com/paper/1808.01348/) (2018)
- [Wasabi: A Framework for Dynamically Analyzing WebAssembly](https://mlsanity.com/paper/1808.10652/) (2018)
- [Not So Fast: Analyzing the Performance of WebAssembly vs. Native Code](https://mlsanity.com/paper/1901.09056/) (2019)
- [TaintAssembly: Taint-Based Information Flow Control Tracking for WebAssembly](https://mlsanity.com/paper/1802.01050/) (2018)
- [Taint Tracking for WebAssembly](https://mlsanity.com/paper/1807.08349/) (2018)

## Abstract (arXiv)

The increasing adoption of WebAssembly (Wasm) for performance-critical and security-sensitive tasks drives the demand for WebAssembly program comprehension and reverse engineering. Recent studies have introduced machine learning (ML)-based WebAssembly reverse engineering tools. Yet, the generalization of task-specific ML solutions remains challenging, because their effectiveness hinges on the availability of an ample supply of high-quality task-specific labeled data. Moreover, previous works overlook the high-level semantics present in source code and its documentation. Acknowledging the abundance of available source code with documentation, which can be compiled into WebAssembly, we propose to learn representations of them concurrently and harness their mutual relationships for effective WebAssembly reverse engineering. In this paper, we present WasmRev, the first multi-modal pre-trained language model for WebAssembly reverse engineering. WasmRev is pre-trained using self-supervised learning on a large-scale multi-modal corpus encompassing source code, code documentation and the compiled WebAssembly, without requiring labeled data. WasmRev incorporates three tailored multi-modal pre-training tasks to capture various characteristics of WebAssembly and cross-modal relationships. WasmRev is only trained once to produce general-purpose representations that can broadly support WebAssembly reverse engineering tasks through few-shot fine-tuning with much less labeled data, improving data efficiency. We fine-tune WasmRev onto three important reverse engineering tasks: type recovery, function purpose identification and WebAssembly summarization. Our results show that WasmRev pre-trained on the corpus of multi-modal samples establishes a robust foundation for these tasks, achieving high task accuracy and outperforming the state-of-the-art ML methods for WebAssembly reverse engineering.