# MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts

arXiv: https://arxiv.org/abs/2406.12549 · HTML: https://mlsanity.com/paper/2406.12549/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 30 of 31)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [TweepFake: about Detecting Deepfake Tweets](https://mlsanity.com/paper/2008.00036/) (2020)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformation](https://mlsanity.com/paper/2310.15515/) (2023)
- [A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators](https://mlsanity.com/paper/2312.15407/) (2023)
- [M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2402.11175/) (2024)
- [Anatomy of an AI-powered malicious social botnet](https://mlsanity.com/paper/2307.16336/) (2023)
- [Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains](https://mlsanity.com/paper/2309.11285/) (2023)
- [Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian](https://mlsanity.com/paper/2206.01583/) (2022)
- [What is Gab? A Bastion of Free Speech or an Alt-Right Echo Chamber?](https://mlsanity.com/paper/1802.05287/) (2018)
- [A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts](https://mlsanity.com/paper/2311.08374/) (2023)
- [Authorship Obfuscation in Multilingual Machine-Generated Text Detection](https://mlsanity.com/paper/2401.07867/) (2024)
- [A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization](https://mlsanity.com/paper/2403.01152/) (2024)
- [METAL: Towards Multilingual Meta-Evaluation](https://mlsanity.com/paper/2404.01667/) (2024)
- [The Pushshift Telegram Dataset](https://mlsanity.com/paper/2001.08438/) (2020)
- [IMGTB: A Framework for Machine-Generated Text Detection Benchmarking](https://mlsanity.com/paper/2311.12574/) (2023)
- [Who Said That? Benchmarking Social Media AI Detection](https://mlsanity.com/paper/2310.08240/) (2023)
- [MAiDE-up: Multilingual Deception Detection of GPT-generated Hotel Reviews](https://mlsanity.com/paper/2404.12938/) (2024)

## Built on by (most cited first, 14 of 14)

- [Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media](https://mlsanity.com/paper/2412.18148/) (2024)
- [Characterizing LLM-driven Social Network: The Chirper.ai Case](https://mlsanity.com/paper/2504.10286/) (2025)
- [mdok of KInIT: Robustly Fine-tuned LLM for Binary and Multiclass AI-Generated Text Detection](https://mlsanity.com/paper/2506.01702/) (2025)
- [A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models](https://mlsanity.com/paper/2410.21360/) (2024)
- [DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models](https://mlsanity.com/paper/2508.00619/) (2025)
- [LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text](https://mlsanity.com/paper/2509.21269/) (2025)
- [Beyond Literal Translation: Evaluating Cultural Effectiveness in Social Media UGC](https://mlsanity.com/paper/2605.25626/) (2026)
- [Beyond speculation: Measuring the growing presence of LLM-generated texts in multilingual disinformation](https://mlsanity.com/paper/2503.23242/) (2025)
- [When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection](https://mlsanity.com/paper/2510.12476/) (2025)
- [When New Generators Arrive: Lifelong Machine-Generated Text Attribution via Ridge Feature Transfer](https://mlsanity.com/paper/2606.05626/) (2026)
- [DuoLens: A Framework for Robust Detection of Machine-Generated Multilingual Text and Code](https://mlsanity.com/paper/2510.18904/) (2025)
- [AEyeDE: An Attention-Based Attribution Framework for AI-Generated Text Detection](https://mlsanity.com/paper/2606.00016/) (2026)
- [On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty Perspective](https://mlsanity.com/paper/2606.02158/) (2026)
- [CrisisFake: Benchmark Validity of AI-Generated Text Detection for Disaster Social Sensing](https://mlsanity.com/paper/2609.35821/) (2026)

## Abstract (arXiv)

Recent LLMs are able to generate high-quality multilingual texts, indistinguishable for humans from authentic human-written ones. Research in machine-generated text detection is however mostly focused on the English language and longer texts, such as news articles, scientific papers or student essays. Social-media texts are usually much shorter and often feature informal language, grammatical errors, or distinct linguistic items (e.g., emoticons, hashtags). There is a gap in studying the ability of existing methods in detection of such texts, reflected also in the lack of existing multilingual benchmark datasets. To fill this gap we propose the first multilingual (22 languages) and multi-platform (5 social media platforms) dataset for benchmarking machine-generated text detection in the social-media domain, called MultiSocial. It contains 472,097 texts, of which about 58k are human-written and approximately the same amount is generated by each of 7 multilingual LLMs. We use this benchmark to compare existing detection methods in zero-shot as well as fine-tuned form. Our results indicate that the fine-tuned detectors have no problem to be trained on social-media texts and that the platform selection for training matters.