# Representation learning with function call graph transformations for malware open set recognition

arXiv: https://arxiv.org/abs/2205.06918 · HTML: https://mlsanity.com/paper/2205.06918/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 13 of 13)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning](https://mlsanity.com/paper/2105.05682/) (2021)
- [Microsoft Malware Classification Challenge](https://mlsanity.com/paper/1802.10135/) (2018)
- [Learning a Neural-network-based Representation for Open Set Recognition](https://mlsanity.com/paper/1802.04365/) (2018)
- [MMF: A loss extension for feature learning in open set recognition](https://mlsanity.com/paper/2006.15117/) (2020)
- [Self-supervised Detransformation Autoencoder for Representation Learning in Open Set Recognition](https://mlsanity.com/paper/2105.13557/) (2021)

## Built on by (most cited first, 2 of 2)

- [Malware families discovery via Open-Set Recognition on Android manifest permissions](https://mlsanity.com/paper/2505.12750/) (2025)
- [Feature Decoupling in Self-supervised Representation Learning for Open Set Recognition](https://mlsanity.com/paper/2209.14385/) (2022)

## Abstract (arXiv)

Open set recognition (OSR) problem has been a challenge in many machine learning (ML) applications, such as security. As new/unknown malware families occur regularly, it is difficult to exhaust samples that cover all the classes for the training process in ML systems. An advanced malware classification system should classify the known classes correctly while sensitive to the unknown class. In this paper, we introduce a self-supervised pre-training approach for the OSR problem in malware classification. We propose two transformations for the function call graph (FCG) based malware representations to facilitate the pretext task. Also, we present a statistical thresholding approach to find the optimal threshold for the unknown class. Moreover, the experiment results indicate that our proposed pre-training process can improve different performances of different downstream loss functions for the OSR problem.