# Malceiver: Perceiver with Hierarchical and Multi-modal Features for Android Malware Detection

arXiv: https://arxiv.org/abs/2204.05994 · HTML: https://mlsanity.com/paper/2204.05994/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 12 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Random Feature Attention](https://mlsanity.com/paper/2103.02143/) (2021)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [Scaling Autoregressive Video Models](https://mlsanity.com/paper/1906.02634/) (2019)
- [Adaptively Sparse Transformers](https://mlsanity.com/paper/1909.00015/) (2019)
- [Malware Detection by Eating a Whole EXE](https://mlsanity.com/paper/1710.09435/) (2017)
- [Deep Learning for Android Malware Defenses: a Systematic Literature Review](https://mlsanity.com/paper/2103.05292/) (2021)
- [R2-D2: ColoR-inspired Convolutional NeuRal Network (CNN)-based AndroiD Malware Detections](https://mlsanity.com/paper/1705.04448/) (2017)
- [HiP: Hierarchical Perceiver](https://mlsanity.com/paper/2202.10890/) (2022)
- [A New Android Malware Detection Approach Using Bayesian Classification](https://mlsanity.com/paper/1608.00848/) (2016)

## Abstract (arXiv)

We propose the Malceiver, a hierarchical Perceiver model for Android malware detection that makes use of multi-modal features. The primary inputs are the opcode sequence and the requested permissions of a given Android APK file. To reach a malware classification decision the model combines hierarchical features extracted from the opcode sequence together with the requested permissions. The model's architecture is based on the Perceiver/PerceiverIO which allows for very long opcode sequences to be processed efficiently. Our proposed model can be easily extended to use multi-modal features. We show experimentally that this model outperforms a conventional CNN architecture for opcode sequence based malware detection. We then show that using additional modalities improves performance. Our proposed architecture opens new avenues for the use of Transformer-style networks in malware research.