# Evaluating Out-of-Distribution Robustness in Graph-Based Android Malware Classification: A New Principled Benchmark

arXiv: https://arxiv.org/abs/2508.06734 · HTML: https://mlsanity.com/paper/2508.06734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Builds on (most cited first, 30 of 32)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [Exphormer: Sparse Transformers for Graphs](https://mlsanity.com/paper/2303.06147/) (2023)
- [GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks](https://mlsanity.com/paper/2302.08043/) (2023)
- [EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models](https://mlsanity.com/paper/1804.04637/) (2018)
- [Universal Prompt Tuning for Graph Neural Networks](https://mlsanity.com/paper/2209.15240/) (2022)
- [Empowering Graph Representation Learning with Test-Time Graph Transformation](https://mlsanity.com/paper/2210.03561/) (2022)
- [A Large-Scale Database for Graph Representation Learning](https://mlsanity.com/paper/2011.07682/) (2020)
- [SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection](https://mlsanity.com/paper/2012.07634/) (2020)
- [CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval](https://mlsanity.com/paper/2411.12644/) (2024)
- [GraphPatcher: Mitigating Degree Bias for Graph Neural Networks via Test-time Augmentation](https://mlsanity.com/paper/2310.00800/) (2023)
- [AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs](https://mlsanity.com/paper/2304.09595/) (2023)
- [G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks](https://mlsanity.com/paper/2305.10329/) (2023)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Edge Prompt Tuning for Graph Neural Networks](https://mlsanity.com/paper/2503.00750/) (2025)
- [MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labels](https://mlsanity.com/paper/2111.15031/) (2021)
- [Test-Time Training for Graph Neural Networks](https://mlsanity.com/paper/2210.08813/) (2022)
- [A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptation](https://mlsanity.com/paper/2410.19265/) (2024)
- [GraphTTA: Test Time Adaptation on Graph Neural Networks](https://mlsanity.com/paper/2208.09126/) (2022)
- [MalNet: A Large-Scale Image Database of Malicious Software](https://mlsanity.com/paper/2102.01072/) (2021)
- [LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis](https://mlsanity.com/paper/2505.18551/) (2025)
- [A Comparison of Graph Neural Networks for Malware Classification](https://mlsanity.com/paper/2303.12812/) (2023)

## Abstract (arXiv)

While graph-based Android malware classifiers report strong benchmark accuracy of over 94%, their performance sharply decreases up to 45% when exposed to previously unseen variants of known malware families. In this work, we systematically investigate this critical yet overlooked challenge for real-world deployment by introducing a benchmarking suite designed to simulate two prevalent scenarios: MalNet-Tiny-Common for covariate shift, and MalNet-Tiny-Distinct for domain shift. We further identify an inherent limitation of existing benchmarks where input representation is limited to structure-only function call graphs, discarding the semantic signals needed for robust cross-distribution reasoning. To verify this, we propose a semantic enrichment framework that extends raw graph topology with function-level attributes, combining lightweight metadata with LLM-based code embeddings. Empirical evaluations confirm the effectiveness of our data-centric methodology, with which classification performs better under distribution shift compared to model-based approaches, and consistently further enhances robustness when used in conjunction. We release our precomputed datasets alongside an extensible pipeline implementation, laying the groundwork for more resilient malware detection systems in evolving threat environments.