# A Novel GPT-Based Framework for Anomaly Detection in System Logs

arXiv: https://arxiv.org/abs/2510.16044 · HTML: https://mlsanity.com/paper/2510.16044/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection](https://mlsanity.com/paper/2107.05908/) (2021)
- [LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model](https://mlsanity.com/paper/2111.09564/) (2021)
- [Interaction-Aware Vulnerability Detection in Smart Contract Bytecodes](https://mlsanity.com/paper/2410.20712/) (2024)
- [Malicious Code Detection in Smart Contracts via Opcode Vectorization](https://mlsanity.com/paper/2504.12720/) (2025)
- [Penetration Testing for System Security: Methods and Practical Approaches](https://mlsanity.com/paper/2505.19174/) (2025)
- [Blockchain security based on cryptography: a review](https://mlsanity.com/paper/2508.01280/) (2025)
- [Blockchain Application in Metaverse: A Review](https://mlsanity.com/paper/2504.11730/) (2025)
- [Mining Characteristics of Vulnerable Smart Contracts Across Lifecycle Stages](https://mlsanity.com/paper/2504.15063/) (2025)
- [AI-Based Vulnerability Analysis of NFT Smart Contracts](https://mlsanity.com/paper/2504.16113/) (2025)
- [When Blockchain Meets Crawlers: Real-time Market Analytics in Solana NFT Markets](https://mlsanity.com/paper/2506.02892/) (2025)
- [Information Security Based on LLM Approaches: A Review](https://mlsanity.com/paper/2507.18215/) (2025)
- [UEChecker: Detecting Unchecked External Call Vulnerabilities in DApps via Graph Analysis](https://mlsanity.com/paper/2508.01343/) (2025)
- [MultiCFV: Detecting Control Flow Vulnerabilities in Smart Contracts Leveraging Multimodal Deep Learning](https://mlsanity.com/paper/2508.01346/) (2025)
- [NATLM: Detecting Defects in NFT Smart Contracts Leveraging LLM](https://mlsanity.com/paper/2508.01351/) (2025)
- [Security Analysis of ChatGPT: Threats and Privacy Risks](https://mlsanity.com/paper/2508.09426/) (2025)
- [Risk Assessment and Security Analysis of Large Language Models](https://mlsanity.com/paper/2508.17329/) (2025)
- [MoveScanner: Analysis of Security Risks of Move Smart Contracts](https://mlsanity.com/paper/2508.17964/) (2025)

## Built on by (most cited first, 1 of 1)

- [Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classification](https://mlsanity.com/paper/2601.07790/) (2026)

## Abstract (arXiv)

Identification of anomalous events within system logs constitutes a pivotal element within the frame- work of cybersecurity defense strategies. However, this process faces numerous challenges, including the management of substantial data volumes, the distribution of anomalies, and the precision of con- ventional methods. To address this issue, the present paper puts forward a proposal for an intelligent detection method for system logs based on Genera- tive Pre-trained Transformers (GPT). The efficacy of this approach is attributable to a combination of structured input design and a Focal Loss op- timization strategy, which collectively result in a substantial enhancement of the performance of log anomaly detection. The initial approach involves the conversion of raw logs into event ID sequences through the use of the Drain parser. Subsequently, the Focal Loss loss function is employed to address the issue of class imbalance. The experimental re- sults demonstrate that the optimized GPT-2 model significantly outperforms the unoptimized model in a range of key metrics, including precision, recall, and F1 score. In specific tasks, comparable or superior performance has been demonstrated to that of the GPT-3.5 API.