mlsanityData to 14 Oct 2026

MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection

arXiv 2103.03806 ↗First posted 5 Mar 2021cs.CR cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

In recent years we have witnessed an increase in cyber threats and malicious software attacks on different platforms with important consequences to persons and businesses. It has become critical to find automated machine learning techniques to proactively defend against malware. Transformers, a category of attention-based deep learning techniques, have recently shown impressive results in solving different tasks mainly related to the field of Natural Language Processing (NLP). In this paper, we propose the use of a Transformers' architecture to automatically detect malicious software. We propose a model based on BERT (Bidirectional Encoder Representations from Transformers) which performs a static analysis on the source code of Android applications using preprocessed features to characterize existing malware and classify it into different representative malware categories. The obtained results are promising and show the high performance obtained by Transformer-based models for malicious software detection.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 13 of its 37 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

13 of 37 references.

References · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2021Transformers in Vision: A Surveycited by 1,013
  9. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  10. 2020Efficient Transformers: A Surveycited by 790
  11. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  12. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
1 more
  1. 2020Java Decompiler Diversity and its Application to Meta-decompilationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 5; 2025: 2; 2026: 1.

12 papers.

  1. 2023SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Securitycited by 37
  2. 2022CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domaincited by 31
  3. 2023On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptionscited by 11
  4. 2024DomURLs_BERT: Pre-trained BERT-based Model for Malicious Domains and URLs Detection and Classificationcited by 5
  5. 2024Ignore Me But Don't Replace Me: Utilizing Non-Linguistic Elements for Pretraining on the Cybersecurity Domaincited by 2
  6. 2024Accelerating Malware Classification: A Vision Transformer Solutioncited by 2
  7. 2025Detection of LLM-Generated Java Code Using Discretized Nested Bigramscited by 2
  8. 2025Empirical Evaluation of Concept Drift in ML-Based Android Malware Detectioncited by 2
  9. 2024SecureReg: Combining NLP and MLP for Enhanced Detection of Malicious Domain Name Registrationscited by 1
  10. 2026Trident: Improving Malware Detection with LLMs and Behavioral Featurescited by 1
  11. 2023The Efficacy of Transformer-based Adversarial Attacks in Security Domainscited by 0
  12. 2024Advancing NLP Security by Leveraging LLMs as Adversarial Enginescited by 0