# LOBO -- Evaluation of Generalization Deficiencies in Twitter Bot Classifiers

arXiv: https://arxiv.org/abs/1809.09684 · HTML: https://mlsanity.com/paper/1809.09684/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2018.

## Built on by (most cited first, 13 of 13)

- [TwiBot-22: Towards Graph-Based Twitter Bot Detection](https://mlsanity.com/paper/2206.04564/) (2022)
- [What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection](https://mlsanity.com/paper/2402.00371/) (2024)
- [Realtime Robust Malicious Traffic Detection via Frequency Domain Analysis](https://mlsanity.com/paper/2106.14707/) (2021)
- [LMBot: Distilling Graph Knowledge into Language Model for Graph-less Deployment in Twitter Bot Detection](https://mlsanity.com/paper/2306.17408/) (2023)
- [Simplistic Collection and Labeling Practices Limit the Utility of Benchmark Datasets for Twitter Bot Detection](https://mlsanity.com/paper/2301.07015/) (2023)
- [Demystifying Misconceptions in Social Bots Research](https://mlsanity.com/paper/2303.17251/) (2023)
- [MulBot: Unsupervised Bot Detection Based on Multivariate Time Series](https://mlsanity.com/paper/2209.10361/) (2022)
- [False Information, Bots and Malicious Campaigns: Demystifying Elements of Social Media Manipulations](https://mlsanity.com/paper/2308.12497/) (2023)
- [Political Communities on Twitter: Case Study of the 2022 French Presidential Election](https://mlsanity.com/paper/2204.07436/) (2022)
- [BSG4Bot: Efficient Bot Detection based on Biased Heterogeneous Subgraphs](https://mlsanity.com/paper/2410.05356/) (2024)
- [BotArtist: Generic approach for bot detection in Twitter via semi-automatic machine learning pipeline](https://mlsanity.com/paper/2306.00037/) (2023)
- [BotSSCL: Social Bot Detection with Self-Supervised Contrastive Learning](https://mlsanity.com/paper/2402.03740/) (2024)
- [Bots Don't Sit Still: A Longitudinal Study of Bot Behaviour Change, Temporal Drift, and Feature-Structure Evolution](https://mlsanity.com/paper/2512.17067/) (2025)

## Abstract (arXiv)

Botnets in online social networks are increasingly often affecting the regular flow of discussion, attacking regular users and their posts, spamming them with irrelevant or offensive content, and even manipulating the popularity of messages and accounts. Researchers and cybercriminals are involved in an arms race, and new and updated botnets designed to defeat current detection systems are constantly developed, rendering such detection systems obsolete. In this paper, we motivate the need for a generalized evaluation in Twitter bot detection and propose a methodology to evaluate bot classifiers by testing them on unseen bot classes. We show that this methodology is empirically robust, using bot classes of varying sizes and characteristics and reaching similar results, and argue that methods trained and tested on single bot classes or datasets might not able to generalize to new bot classes. We train one such classifier on over 200,000 data points and show that it achieves over 97% accuracy. The data used to train and test this classifier includes some of the largest and most varied collections of bots used in literature. We then test this theoretically sound classifier using our methodology, highlighting that it does not generalize well to unseen bot classes. Finally, we discuss the implications of our results, and reasons why some bot classes are easier and faster to detect than others.