# Nebula: Self-Attention for Dynamic Malware Analysis

arXiv: https://arxiv.org/abs/2310.10664 · HTML: https://mlsanity.com/paper/2310.10664/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 71 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Dynamic Malware Analysis with Feature Engineering and Feature Learning](https://mlsanity.com/paper/1907.07352/) (2019)
- [eXpose: A Character-Level Convolutional Neural Network with Embeddings For Detecting Malicious URLs, File Paths and Registry Keys](https://mlsanity.com/paper/1702.08568/) (2017)
- [SoK: The Impact of Unlabelled Data in Cyberthreat Detection](https://mlsanity.com/paper/2205.08944/) (2022)
- [Neurlux: Dynamic Malware Analysis Without Feature Engineering](https://mlsanity.com/paper/1910.11376/) (2019)
- [Quo Vadis: Hybrid Machine Learning Meta-Model based on Contextual and Behavioral Malware Representations](https://mlsanity.com/paper/2208.12248/) (2022)
- [I-MAD: Interpretable Malware Detector Using Galaxy Transformer](https://mlsanity.com/paper/1909.06865/) (2019)
- [Transformers for End-to-End InfoSec Tasks: A Feasibility Study](https://mlsanity.com/paper/2212.02666/) (2022)
- [An Ensemble of Pre-trained Transformer Models For Imbalanced Multiclass Malware Classification](https://mlsanity.com/paper/2112.13236/) (2021)
- [Shell Language Processing: Unix command parsing for Machine Learning](https://mlsanity.com/paper/2107.02438/) (2021)
- [Mill.jl and JsonGrinder.jl: automated differentiable feature extraction for learning from raw JSON data](https://mlsanity.com/paper/2105.09107/) (2021)
- [Avast-CTU Public CAPE Dataset](https://mlsanity.com/paper/2209.03188/) (2022)

## Built on by (most cited first, 8 of 8)

- [SLIFER: Investigating Performance and Robustness of Malware Detection Pipelines](https://mlsanity.com/paper/2405.14478/) (2024)
- [Robust Synthetic Data-Driven Detection of Living-Off-the-Land Reverse Shells](https://mlsanity.com/paper/2402.18329/) (2024)
- [Trident: Improving Malware Detection with LLMs and Behavioral Features](https://mlsanity.com/paper/2605.00297/) (2026)
- [BEACON: Behavioral Malware Classification with Large Language Model Embeddings and Deep Learning](https://mlsanity.com/paper/2509.14519/) (2025)
- [One Detector Fits All: Robust and Adaptive Detection of Malicious Packages from PyPI to Enterprises](https://mlsanity.com/paper/2512.04338/) (2025)
- [MeLeMaD: Adaptive Malware Detection via Chunk-wise Feature Selection and Meta-Learning](https://mlsanity.com/paper/2512.23987/) (2025)
- [Self-Supervised Representations for Binary Program Clustering: From Empirical Study to Retrieval-Augmented Learning](https://mlsanity.com/paper/2608.02348/) (2026)
- [Bytes to Threats: Byte-Sequence Transformer For Malware Detection](https://mlsanity.com/paper/2610.18787/) (2026)

## Abstract (arXiv)

Dynamic analysis enables detecting Windows malware by executing programs in a controlled environment and logging their actions. Previous work has proposed training machine learning models, i.e., convolutional and long short-term memory networks, on homogeneous input features like runtime APIs to either detect or classify malware, neglecting other relevant information coming from heterogeneous data like network and file operations. To overcome these issues, we introduce Nebula, a versatile, self-attention Transformer-based neural architecture that generalizes across different behavioral representations and formats, combining diverse information from dynamic log reports. Nebula is composed by several components needed to tokenize, filter, normalize and encode data to feed the transformer architecture. We firstly perform a comprehensive ablation study to evaluate their impact on the performance of the whole system, highlighting which components can be used as-is, and which must be enriched with specific domain knowledge. We perform extensive experiments on both malware detection and classification tasks, using three datasets acquired from different dynamic analyses platforms, show that, on average, Nebula outperforms state-of-the-art models at low false positive rates, with a peak of 12% improvement. Moreover, we showcase how self-supervised learning pre-training matches the performance of fully-supervised models with only 20% of training data, and we inspect the output of Nebula through explainable AI techniques, pinpointing how attention is focusing on specific tokens correlated to malicious activities of malware families. To foster reproducibility, we open-source our findings and models at https://github.com/dtrizna/nebula.