# CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning

arXiv: https://arxiv.org/abs/2212.10341 · HTML: https://mlsanity.com/paper/2212.10341/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 73 references that are in the map.
- First posted on arXiv 20 Dec 2022.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Probing Neural Network Comprehension of Natural Language Arguments](https://mlsanity.com/paper/1907.07355/) (2019)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [Real or Fake? Learning to Discriminate Machine from Human Generated Text](https://mlsanity.com/paper/1906.03351/) (2019)
- [Do Massively Pretrained Language Models Make Better Storytellers?](https://mlsanity.com/paper/1909.10705/) (2019)
- [Semi-supervised sequence tagging with bidirectional language models](https://mlsanity.com/paper/1705.00108/) (2017)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection](https://mlsanity.com/paper/1907.09177/) (2019)

## Built on by (most cited first, 30 of 33)

- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [Multiscale Positive-Unlabeled Detection of AI-Generated Texts](https://mlsanity.com/paper/2305.18149/) (2023)
- [M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2402.11175/) (2024)
- [GPT-who: An Information Density-based Machine-Generated Text Detector](https://mlsanity.com/paper/2310.06202/) (2023)
- [SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection](https://mlsanity.com/paper/2404.14183/) (2024)
- [Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers](https://mlsanity.com/paper/2401.06461/) (2024)
- [Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks](https://mlsanity.com/paper/2402.11638/) (2024)
- [Permute-and-Flip: An optimally stable and watermarkable decoder for LLMs](https://mlsanity.com/paper/2402.05864/) (2024)
- [Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?](https://mlsanity.com/paper/2304.01002/) (2023)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs](https://mlsanity.com/paper/2402.10586/) (2024)
- [A Simple yet Efficient Ensemble Approach for AI-generated Text Detection](https://mlsanity.com/paper/2311.03084/) (2023)
- [On the Zero-Shot Generalization of Machine-Generated Text Detectors](https://mlsanity.com/paper/2310.05165/) (2023)
- [Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection](https://mlsanity.com/paper/2510.08602/) (2025)
- [ToBlend: Token-Level Blending With an Ensemble of LLMs to Attack AI-Generated Text Detection](https://mlsanity.com/paper/2402.11167/) (2024)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [PetKaz at SemEval-2024 Task 8: Can Linguistics Capture the Specifics of LLM-generated Text?](https://mlsanity.com/paper/2404.05483/) (2024)
- [Detection of Fake Generated Scientific Abstracts](https://mlsanity.com/paper/2304.06148/) (2023)
- [Stacking the Odds: Transformer-Based Ensemble for AI-Generated Text Detection](https://mlsanity.com/paper/2310.18906/) (2023)
- [TM-TREK at SemEval-2024 Task 8: Towards LLM-Based Automatic Boundary Detection for Human-Machine Mixed Text](https://mlsanity.com/paper/2404.00899/) (2024)
- [AISPACE at SemEval-2024 task 8: A Class-balanced Soft-voting System for Detecting Multi-generator Machine-generated Text](https://mlsanity.com/paper/2404.00950/) (2024)
- [Tracing Thought: Using Chain-of-Thought Reasoning to Identify the LLM Behind AI-Generated Text](https://mlsanity.com/paper/2504.16913/) (2025)
- [Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models](https://mlsanity.com/paper/2601.04448/) (2026)
- [Style-News: Incorporating Stylized News Generation and Adversarial Verification for Neural Fake News Detection](https://mlsanity.com/paper/2401.15509/) (2024)
- [Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection](https://mlsanity.com/paper/2602.08031/) (2026)

## Abstract (arXiv)

Machine-Generated Text (MGT) detection, a task that discriminates MGT from Human-Written Text (HWT), plays a crucial role in preventing misuse of text generative models, which excel in mimicking human writing style recently. Latest proposed detectors usually take coarse text sequences as input and fine-tune pretrained models with standard cross-entropy loss. However, these methods fail to consider the linguistic structure of texts. Moreover, they lack the ability to handle the low-resource problem which could often happen in practice considering the enormous amount of textual data online. In this paper, we present a coherence-based contrastive learning model named CoCo to detect the possible MGT under low-resource scenario. To exploit the linguistic feature, we encode coherence information in form of graph into text representation. To tackle the challenges of low data resource, we employ a contrastive learning framework and propose an improved contrastive loss for preventing performance degradation brought by simple samples. The experiment results on two public datasets and two self-constructed datasets prove our approach outperforms the state-of-art methods significantly. Also, we surprisingly find that MGTs originated from up-to-date language models could be easier to detect than these from previous models, in our experiments. And we propose some preliminary explanations for this counter-intuitive phenomena. All the codes and datasets are open-sourced.