# An Empirical Study of IoT Security Aspects at Sentence-Level in Developer Textual Discussions

arXiv: https://arxiv.org/abs/2206.03079 · HTML: https://mlsanity.com/paper/2206.03079/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 88 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Code and Named Entity Recognition in StackOverflow](https://mlsanity.com/paper/2005.01634/) (2020)
- [Practical Text Classification With Large Pre-Trained Language Models](https://mlsanity.com/paper/1812.01207/) (2018)
- [BERT-Based Sentiment Analysis: A Software Engineering Perspective](https://mlsanity.com/paper/2106.02581/) (2021)
- [Soteria: Automated IoT Safety and Security Analysis](https://mlsanity.com/paper/1805.08876/) (2018)
- [Cross-App Interference Threats in Smart Homes: Categorization, Detection and Handling](https://mlsanity.com/paper/1808.02125/) (2018)
- [Development and Application of Sentiment Analysis Tools in Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2105.02703/) (2021)
- [An Empirical Study of the Effectiveness of an Ensemble of Stand-alone Sentiment Detection Tools for Software Engineering Datasets](https://mlsanity.com/paper/2111.03196/) (2021)
- [Sensitive Information Tracking in Commodity IoT](https://mlsanity.com/paper/1802.08307/) (2018)
- [PIANO: Proximity-based User Authentication on Voice-Powered Internet-of-Things Devices](https://mlsanity.com/paper/1704.03118/) (2017)
- [Security and Machine Learning Adoption in IoT: A Preliminary Study of IoT Developer Discussions](https://mlsanity.com/paper/2104.00634/) (2021)

## Built on by (most cited first, 3 of 3)

- [Can Transformer Models Effectively Detect Software Aspects in StackOverflow Discussion?](https://mlsanity.com/paper/2209.12065/) (2022)
- [An Empirical Study on How the Developers Discussed about Pandas Topics](https://mlsanity.com/paper/2210.03519/) (2022)
- [Effectiveness of Transformer Models on IoT Security Detection in StackOverflow Discussions](https://mlsanity.com/paper/2207.14542/) (2022)

## Abstract (arXiv)

IoT is a rapidly emerging paradigm that now encompasses almost every aspect of our modern life. As such, ensuring the security of IoT devices is crucial. IoT devices can differ from traditional computing, thereby the design and implementation of proper security measures can be challenging in IoT devices. We observed that IoT developers discuss their security-related challenges in developer forums like Stack Overflow(SO). However, we find that IoT security discussions can also be buried inside non-security discussions in SO. In this paper, we aim to understand the challenges IoT developers face while applying security practices and techniques to IoT devices. We have two goals: (1) Develop a model that can automatically find security-related IoT discussions in SO, and (2) Study the model output to learn about IoT developer security-related challenges. First, we download 53K posts from SO that contain discussions about IoT. Second, we manually labeled 5,919 sentences from 53K posts as 1 or 0. Third, we use this benchmark to investigate a suite of deep learning transformer models. The best performing model is called SecBot. Fourth, we apply SecBot on the entire posts and find around 30K security related sentences. Fifth, we apply topic modeling to the security-related sentences. Then we label and categorize the topics. Sixth, we analyze the evolution of the topics in SO. We found that (1) SecBot is based on the retraining of the deep learning model RoBERTa. SecBot offers the best F1-Score of 0.935, (2) there are six error categories in misclassified samples by SecBot. SecBot was mostly wrong when the keywords/contexts were ambiguous (e.g., gateway can be a security gateway or a simple gateway), (3) there are 9 security topics grouped into three categories: Software, Hardware, and Network, and (4) the highest number of topics belongs to software security, followed by network security.