mlsanityData to 14 Oct 2026

Holographic Global Convolutional Networks for Long-Range Prediction Tasks in Malware Detection

arXiv 2403.17978 ↗First posted 23 Mar 2024cs.CR cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Malware detection is an interesting and valuable domain to work in because it has significant real-world impact and unique machine-learning challenges. We investigate existing long-range techniques and benchmarks and find that they're not very suitable in this problem area. In this paper, we introduce Holographic Global Convolutional Networks (HGConv) that utilize the properties of Holographic Reduced Representations (HRR) to encode and decode features from sequence elements. Unlike other global convolutional methods, our method does not require any intricate kernel computation or crafted kernel design. HGConv kernels are defined as simple parameters learned through backpropagation. The proposed method has achieved new SOTA results on Microsoft Malware Classification Challenge, Drebin, and EMBER malware benchmarks. With log-linear complexity in sequence length, the empirical results demonstrate substantially faster run-time by HGConv compared to other methods achieving far more efficient scaling even with sequence length ≥ 100,000.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 43 of its 80 references that are in the map. First posted on arXiv 23 Mar 2024.

Builds on

43 of 80 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2016Layer Normalizationcited by 5,669
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  12. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
31 more
  1. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  2. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  3. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  4. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  5. 2016Language Modeling with Gated Convolutional Networkscited by 702
  6. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  7. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  8. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  9. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
  10. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  11. 2021FNet: Mixing Tokens with Fourier Transformscited by 347
  12. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 194
  13. 2018EMBER: An Open Dataset for Training Static PE Malware Machine Learning Modelscited by 165
  14. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  15. 2018Learning long-range spatial dependencies with horizontal gated-recurrent unitscited by 98
  16. 2021Luna: Linear Unified Nested Attentioncited by 96
  17. 2022What Makes Convolutional Models Great on Long Sequence Modeling?cited by 87
  18. 2021H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequencescited by 47
  19. 2020A Survey of Machine Learning Methods and Challenges for Windows Malware Classificationcited by 33
  20. 2020Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detectioncited by 24
  21. 2023Spiking Structured State Space Model for Monaural Speech Enhancementcited by 23
  22. 2020Bringing UMAP Closer to the Speed of Light with GPU Accelerationcited by 19
  23. 2023Recasting Self-Attention with Holographic Reduced Representationscited by 14
  24. 2023MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superpositioncited by 14
  25. 2023DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attentioncited by 10
  26. 2017Lempel-Ziv Jaccard Distance, an Effective Alternative to Ssdeep and Sdhashcited by 8
  27. 2022Deploying Convolutional Networks on Untrusted Platforms Using 2D Holographic Reduced Representationscited by 7
  28. 2019A New Burrows Wheeler Transform Markov Distancecited by 6
  29. 2023DNArch: Learning Convolutional Neural Architectures by Backpropagationcited by 4
  30. 2023Towards Generalization in Subitizing with Neuro-Symbolic Loss using Holographic Reduced Representationscited by 4
  31. 2022Lempel-Ziv Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 179
  2. 2024Is Function Similarity Over-Engineered? Building a Benchmarkcited by 6
  3. 2024A Walsh Hadamard Derived Linear Vector Symbolic Architecturecited by 4
  4. 2025Linearithmic Clean-up for Vector-Symbolic Key-Value Memory with Kroneker Rotation Productscited by 2
  5. 2026Cybersecurity is the True Frontier for Generative AI Success or Failurecited by 2
  6. 2025A Vector Symbolic Approach to Multiple Instance Learningcited by 0