# Evaluating Language Models For Threat Detection in IoT Security Logs

arXiv: https://arxiv.org/abs/2507.02390 · HTML: https://mlsanity.com/paper/2507.02390/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 13 of 13)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Towards the Development of Realistic Botnet Dataset in the Internet of Things for Network Forensic Analytics: Bot-IoT Dataset](https://mlsanity.com/paper/1811.00701/) (2018)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices](https://mlsanity.com/paper/2306.14263/) (2023)
- [LogGPT: Log Anomaly Detection via GPT](https://mlsanity.com/paper/2309.14482/) (2023)
- [Benchmarking Large Language Models for Log Analysis, Security, and Interpretation](https://mlsanity.com/paper/2311.14519/) (2023)
- [LLM-based event log analysis techniques: A survey](https://mlsanity.com/paper/2502.00677/) (2025)
- [Security Vulnerability Detection with Multitask Self-Instructed Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2406.05892/) (2024)
- [LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis](https://mlsanity.com/paper/2407.01896/) (2024)
- [LogPr'ecis: Unleashing Language Models for Automated Malicious Log Analysis](https://mlsanity.com/paper/2307.08309/) (2023)
- [RedChronos: A Large Language Model-Based Log Analysis System for Insider Threat Detection in Enterprises](https://mlsanity.com/paper/2503.02702/) (2025)
- [LLM meets ML: Data-efficient Anomaly Detection on Unstable Logs](https://mlsanity.com/paper/2406.07467/) (2024)

## Built on by (most cited first, 4 of 4)

- [CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alerts](https://mlsanity.com/paper/2603.04186/) (2026)
- [Information-Dense Reasoning for Efficient and Auditable Security Alert Triage](https://mlsanity.com/paper/2512.08169/) (2025)
- [Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injection](https://mlsanity.com/paper/2607.24174/) (2026)
- [CESBench: Benchmarking Large Language Models on Cryptographic Engineering Security for IoT Devices](https://mlsanity.com/paper/2609.21344/) (2026)

## Abstract (arXiv)

Log analysis is a relevant research field in cybersecurity as they can provide a source of information for the detection of threats to networks and systems. This paper presents a pipeline to use fine-tuned Large Language Models (LLMs) for anomaly detection and mitigation recommendation using IoT security logs. Utilizing classical machine learning classifiers as a baseline, three open-source LLMs are compared for binary and multiclass anomaly detection, with three strategies: zero-shot, few-shot prompting and fine-tuning using an IoT dataset. LLMs give better results on multi-class attack classification than the corresponding baseline models. By mapping detected threats to MITRE CAPEC, defining a set of IoT-specific mitigation actions, and fine-tuning the models with those actions, the models are able to provide a combined detection and recommendation guidance.