# Bytes to Threats: Byte-Sequence Transformer For Malware Detection

arXiv: https://arxiv.org/abs/2610.18787 · HTML: https://mlsanity.com/paper/2610.18787/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Malware Detection by Eating a Whole EXE](https://mlsanity.com/paper/1710.09435/) (2017)
- [Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection](https://mlsanity.com/paper/2012.09390/) (2020)
- [Bytes Are All You Need: Transformers Operating Directly On File Bytes](https://mlsanity.com/paper/2306.00238/) (2023)
- [Deep learning at the shallow end: Malware classification for non-domain experts](https://mlsanity.com/paper/1807.08265/) (2018)
- [I-MAD: Interpretable Malware Detector Using Galaxy Transformer](https://mlsanity.com/paper/1909.06865/) (2019)
- [Reliable Detection of Compressed and Encrypted Data](https://mlsanity.com/paper/2103.17059/) (2021)
- [Nebula: Self-Attention for Dynamic Malware Analysis](https://mlsanity.com/paper/2310.10664/) (2023)
- [Self-Supervised Vision Transformers for Malware Detection](https://mlsanity.com/paper/2208.07049/) (2022)
- [AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regression](https://mlsanity.com/paper/2403.13565/) (2024)
- [SoK: Leveraging Transformers for Malware Analysis](https://mlsanity.com/paper/2405.17190/) (2024)

## Abstract (arXiv)

Existing malware detection approaches heavily rely on handcrafted features derived from static or dynamic analysis of the executable files. These methods require specialized domain expertise, dedicated analysis tools, and time-consuming analysis, which limits their scalability and robustness in the face of the increasing complexity and diversity of modern malware. Consequently, there is a growing interest in approaches that operate directly on raw bytes, which contain rich semantic and structural information. However, capturing long-range dependencies within executable files remains challenging for conventional static and CNN-based methods. This paper presents MalwareFormer, a transformer-based, byte-level malware detection framework that operates directly on raw bytes, without requiring domain knowledge or feature extraction. MalwareFormer achieves an accuracy of 98.35%, a balanced accuracy of 98.32%, a macro-F1 of 98.32%, and a false positive rate of 1.94%, outperforming both baselines across all threshold-dependent classification metrics and demonstrating that transformers provide a powerful and scalable approach for byte-level malware detection.