# On Improving Deep Learning Trace Analysis with System Call Arguments

arXiv: https://arxiv.org/abs/2103.06915 · HTML: https://mlsanity.com/paper/2103.06915/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection](https://mlsanity.com/paper/1803.04967/) (2018)
- [LSTM-Based System-Call Language Modeling and Robust Ensemble Method for Designing Host-Based Intrusion Detection Systems](https://mlsanity.com/paper/1611.01726/) (2016)
- [Process Monitoring on Sequences of System Call Count Vectors](https://mlsanity.com/paper/1707.03821/) (2017)
- [Intrusion Prediction with System-call Sequence-to-Sequence Model](https://mlsanity.com/paper/1808.01717/) (2018)
- [Automatic Cause Detection of Performance Problems in Web Applications](https://mlsanity.com/paper/2103.04954/) (2021)

## Built on by (most cited first, 3 of 3)

- [A Practical Survey on Faster and Lighter Transformers](https://mlsanity.com/paper/2103.14636/) (2021)
- [Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey](https://mlsanity.com/paper/2408.07583/) (2024)
- [Language Models for Novelty Detection in System Call Traces](https://mlsanity.com/paper/2309.02206/) (2023)

## Abstract (arXiv)

Kernel traces are sequences of low-level events comprising a name and multiple arguments, including a timestamp, a process id, and a return value, depending on the event. Their analysis helps uncover intrusions, identify bugs, and find latency causes. However, their effectiveness is hindered by omitting the event arguments. To remedy this limitation, we introduce a general approach to learning a representation of the event names along with their arguments using both embedding and encoding. The proposed method is readily applicable to most neural networks and is task-agnostic. The benefit is quantified by conducting an ablation study on three groups of arguments: call-related, process-related, and time-related. Experiments were conducted on a novel web request dataset and validated on a second dataset collected on pre-production servers by Ciena, our partnering company. By leveraging additional information, we were able to increase the performance of two widely-used neural networks, an LSTM and a Transformer, by up to 11.3% on two unsupervised language modelling tasks. Such tasks may be used to detect anomalies, pre-train neural networks to improve their performance, and extract a contextual representation of the events.