# Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection

arXiv: https://arxiv.org/abs/2012.09390 · HTML: https://mlsanity.com/paper/2012.09390/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2020.

## Built on by (most cited first, 24 of 24)

- [MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labels](https://mlsanity.com/paper/2111.15031/) (2021)
- [RS-Del: Edit Distance Robustness Certificates for Sequence Classifiers via Randomized Deletion](https://mlsanity.com/paper/2302.01757/) (2023)
- [Recasting Self-Attention with Holographic Reduced Representations](https://mlsanity.com/paper/2305.19534/) (2023)
- [MalCL: Leveraging GAN-Based Generative Replay to Combat Catastrophic Forgetting in Malware Classification](https://mlsanity.com/paper/2501.01110/) (2025)
- [Holographic Global Convolutional Networks for Long-Range Prediction Tasks in Malware Detection](https://mlsanity.com/paper/2403.17978/) (2024)
- [Is Function Similarity Over-Engineered? Building a Benchmark](https://mlsanity.com/paper/2410.22677/) (2024)
- [Assessing the Impact of Packing on Machine Learning-Based Malware Detection and Classification Systems](https://mlsanity.com/paper/2410.24017/) (2024)
- [Small Effect Sizes in Malware Detection? Make Harder Train/Test Splits!](https://mlsanity.com/paper/2312.15813/) (2023)
- [Evading Malware Classifiers via Monte Carlo Mutant Feature Discovery](https://mlsanity.com/paper/2106.07860/) (2021)
- [Cybersecurity is the True Frontier for Generative AI Success or Failure](https://mlsanity.com/paper/2606.28929/) (2026)
- [Toward the Detection of Polyglot Files](https://mlsanity.com/paper/2203.07561/) (2022)
- [SeqNet: An Efficient Neural Network for Automatic Malware Detection](https://mlsanity.com/paper/2205.03850/) (2022)
- [Explainable Malware Detection with Tailored Logic Explained Networks](https://mlsanity.com/paper/2405.03009/) (2024)
- [A Progressive Transformer for Unifying Binary Code Embedding and Knowledge Transfer](https://mlsanity.com/paper/2412.11177/) (2024)
- [A Comparison of State-of-the-Art Techniques for Generating Adversarial Malware Binaries](https://mlsanity.com/paper/2111.11487/) (2021)
- [FGAM:Fast Adversarial Malware Generation Method Based on Gradient Sign](https://mlsanity.com/paper/2305.12770/) (2023)
- [Certified Adversarial Robustness of Machine Learning-based Malware Detectors via (De)Randomized Smoothing](https://mlsanity.com/paper/2405.00392/) (2024)
- [A Cross-Lingual Meta-Learning Method Based on Domain Adaptation for Speech Emotion Recognition](https://mlsanity.com/paper/2410.04633/) (2024)
- [ASSEMBLAGE-DEEPHISTORY: A Cross-Build Binary Dataset with Temporal Coverage](https://mlsanity.com/paper/2605.21615/) (2026)
- [Large Byte Model: Teaching Language Models About Compiled Code](https://mlsanity.com/paper/2606.02834/) (2026)
- [SoK: AI-Augmented Binary Reversing](https://mlsanity.com/paper/2606.17398/) (2026)
- [Delphi Scanner: efficient and interpretable static malware detection via API sequence modeling](https://mlsanity.com/paper/2609.19900/) (2026)
- [Fast And Accurate Text Content File Type Identification](https://mlsanity.com/paper/2609.21306/) (2026)
- [Bytes to Threats: Byte-Sequence Transformer For Malware Detection](https://mlsanity.com/paper/2610.18787/) (2026)

## Abstract (arXiv)

Recent works within machine learning have been tackling inputs of ever-increasing size, with cybersecurity presenting sequence classification problems of particularly extreme lengths. In the case of Windows executable malware detection, inputs may exceed 100 MB, which corresponds to a time series with T=100,000,000 steps. To date, the closest approach to handling such a task is MalConv, a convolutional neural network capable of processing up to T=2,000,000 steps. The O(T) memory of CNNs has prevented further application of CNNs to malware. In this work, we develop a new approach to temporal max pooling that makes the required memory invariant to the sequence length T. This makes MalConv 116× more memory efficient, and up to 25.8× faster to train on its original dataset, while removing the input length restrictions to MalConv. We re-invest these gains into improving the MalConv architecture by developing a new Global Channel Gating design, giving us an attention mechanism capable of learning feature interactions across 100 million time steps in an efficient manner, a capability lacked by the original MalConv CNN. Our implementation can be found at https://github.com/NeuromorphicComputationResearchProgram/MalConv2