# M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection

arXiv: https://arxiv.org/abs/2402.11175 · HTML: https://mlsanity.com/paper/2402.11175/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 17 Feb 2024.

## Builds on (most cited first, 30 of 35)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models](https://mlsanity.com/paper/2111.02840/) (2021)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [Protecting Language Generation Models via Invisible Watermarking](https://mlsanity.com/paper/2302.03162/) (2023)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples](https://mlsanity.com/paper/2307.11729/) (2023)
- [RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting](https://mlsanity.com/paper/2305.15685/) (2023)
- [Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text](https://mlsanity.com/paper/2212.12672/) (2022)
- [Stylometric Detection of AI-Generated Text in Twitter Timelines](https://mlsanity.com/paper/2303.03697/) (2023)
- [The Next Chapter: A Study of Large Language Models in Storytelling](https://mlsanity.com/paper/2301.09790/) (2023)
- [Neural Deepfake Detection with Factual Structure of Text](https://mlsanity.com/paper/2010.07475/) (2020)
- [GPT-who: An Information Density-based Machine-Generated Text Detector](https://mlsanity.com/paper/2310.06202/) (2023)
- [Few-Shot Detection of Machine-Generated Text using Style Representations](https://mlsanity.com/paper/2401.06712/) (2024)

## Built on by (most cited first, 30 of 43)

- [SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection](https://mlsanity.com/paper/2404.14183/) (2024)
- [Beemo: Benchmark of Expert-edited Machine-generated Outputs](https://mlsanity.com/paper/2411.04032/) (2024)
- [LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection](https://mlsanity.com/paper/2408.04284/) (2024)
- [Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards](https://mlsanity.com/paper/2501.07493/) (2025)
- [MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts](https://mlsanity.com/paper/2406.12549/) (2024)
- [Machine-generated text detection prevents language model collapse](https://mlsanity.com/paper/2502.15654/) (2025)
- [GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human](https://mlsanity.com/paper/2501.11012/) (2025)
- [DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning](https://mlsanity.com/paper/2510.17489/) (2025)
- [On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing](https://mlsanity.com/paper/2412.17242/) (2024)
- [Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors](https://mlsanity.com/paper/2505.24523/) (2025)
- [Detection of Human and Machine-Authored Fake News in Urdu](https://mlsanity.com/paper/2410.19517/) (2024)
- [KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis](https://mlsanity.com/paper/2503.00032/) (2025)
- [Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity](https://mlsanity.com/paper/2407.13105/) (2024)
- [VaxGuard: A Multi-Generator, Multi-Type, and Multi-Role Dataset for Detecting LLM-Generated Vaccine Misinformation](https://mlsanity.com/paper/2503.09103/) (2025)
- [Synthetic Lyrics Detection Across Languages and Genres](https://mlsanity.com/paper/2406.15231/) (2024)
- [Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applications](https://mlsanity.com/paper/2502.00808/) (2025)
- [Beyond Easy Wins: A Text Hardness-Aware Benchmark for LLM-generated Text Detection](https://mlsanity.com/paper/2507.15286/) (2025)
- [A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models](https://mlsanity.com/paper/2410.21360/) (2024)
- [Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection](https://mlsanity.com/paper/2501.03940/) (2025)
- [Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection](https://mlsanity.com/paper/2502.12611/) (2025)
- [Robust and Fine-Grained Detection of AI Generated Texts](https://mlsanity.com/paper/2504.11952/) (2025)
- [PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning](https://mlsanity.com/paper/2506.15683/) (2025)
- [On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?](https://mlsanity.com/paper/2510.20810/) (2025)
- [Sharif-MGTD at SemEval-2024 Task 8: A Transformer-Based Approach to Detect Machine Generated Text](https://mlsanity.com/paper/2407.11774/) (2024)
- [RKadiyala at SemEval-2024 Task 8: Black-Box Word-Level Text Boundary Detection in Partially Machine Generated Texts](https://mlsanity.com/paper/2410.16659/) (2024)
- [LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text](https://mlsanity.com/paper/2509.21269/) (2025)
- [Machine Text Detectors are Membership Inference Attacks](https://mlsanity.com/paper/2510.19492/) (2025)
- [Who Wrote the Book? Detecting and Attributing LLM Ghostwriters](https://mlsanity.com/paper/2603.28054/) (2026)
- [TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices](https://mlsanity.com/paper/2605.31113/) (2026)
- [MasonTigers at SemEval-2024 Task 8: Performance Analysis of Transformer-based Models on Machine-Generated Text Detection](https://mlsanity.com/paper/2403.14989/) (2024)

## Abstract (arXiv)

The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels. This raises legitimate concerns about its potential misuse and societal implications. The need to identify and differentiate such content from genuine human-generated text is critical in combating disinformation, preserving the integrity of education and scientific fields, and maintaining trust in communication. In this work, we address this problem by introducing a new benchmark based on a multilingual, multi-domain, and multi-generator corpus of MGTs -- M4GT-Bench. The benchmark is compiled of three tasks: (1) mono-lingual and multi-lingual binary MGT detection; (2) multi-way detection where one need to identify, which particular model generated the text; and (3) mixed human-machine text detection, where a word boundary delimiting MGT from human-written content should be determined. On the developed benchmark, we have tested several MGT detection baselines and also conducted an evaluation of human performance. We see that obtaining good performance in MGT detection usually requires an access to the training data from the same domain and generators. The benchmark is available at https://github.com/mbzuai-nlp/M4GT-Bench.