# Cracking IoT Security: Can LLMs Outsmart Static Analysis Tools?

arXiv: https://arxiv.org/abs/2601.00559 · HTML: https://mlsanity.com/paper/2601.00559/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 2 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [Examining Zero-Shot Vulnerability Repair with Large Language Models](https://mlsanity.com/paper/2112.02125/) (2021)
- [Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants](https://mlsanity.com/paper/2208.09727/) (2022)
- [A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification](https://mlsanity.com/paper/2305.14752/) (2023)
- [When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?](https://mlsanity.com/paper/2309.05520/) (2023)
- [Do you still need a manual smart contract audit?](https://mlsanity.com/paper/2306.12338/) (2023)
- [Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection](https://mlsanity.com/paper/2407.16235/) (2024)
- [SecureFalcon: Are We There Yet in Automated Software Vulnerability Detection with LLMs?](https://mlsanity.com/paper/2307.06616/) (2023)
- [The Emergence of Large Language Models in Static Analysis: A First Look through Micro-Benchmarks](https://mlsanity.com/paper/2402.17679/) (2024)
- [Soteria: Automated IoT Safety and Security Analysis](https://mlsanity.com/paper/1805.08876/) (2018)
- [IoTSan: Fortifying the Safety of IoT Systems](https://mlsanity.com/paper/1810.09551/) (2018)
- [Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats](https://mlsanity.com/paper/2410.22293/) (2024)
- [VISCR: Intuitive & Conflict-free Automation for Securing the Dynamic Consumer IoT Infrastructures](https://mlsanity.com/paper/1907.13288/) (2019)

## Built on by (most cited first, 1 of 1)

- [SmartHomeSecure: Automated Detection and Repair of Smart Home Configuration Errors Using Large Language Models](https://mlsanity.com/paper/2607.06748/) (2026)

## Abstract (arXiv)

Smart home IoT platforms such as openHAB rely on Trigger Action Condition (TAC) rules to automate device behavior, but the interplay among these rules can give rise to interaction threats, unintended or unsafe behaviors emerging from implicit dependencies, conflicting triggers, or overlapping conditions. Identifying these threats requires semantic understanding and structural reasoning that traditionally depend on symbolic, constraint-driven static analysis. This work presents the first comprehensive evaluation of Large Language Models (LLMs) across a multi-category interaction threat taxonomy, assessing their performance on both the original openHAB (oHC/IoTB) dataset and a structurally challenging Mutation dataset designed to test robustness under rule transformations. We benchmark Llama 3.1 8B, Llama 70B, GPT-4o, Gemini-2.5-Pro, and DeepSeek-R1 across zero-, one-, and two-shot settings, comparing their results against oHIT's manually validated ground truth. Our findings show that while LLMs exhibit promising semantic understanding, particularly on action- and condition-related threats, their accuracy degrades significantly for threats requiring cross-rule structural reasoning, especially under mutated rule forms. Model performance varies widely across threat categories and prompt settings, with no model providing consistent reliability. In contrast, the symbolic reasoning baseline maintains stable detection across both datasets, unaffected by rule rewrites or structural perturbations. These results underscore that LLMs alone are not yet dependable for safety critical interaction-threat detection in IoT environments. We discuss the implications for tool design and highlight the potential of hybrid architectures that combine symbolic analysis with LLM-based semantic interpretation to reduce false positives while maintaining structural rigor.