# Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection

arXiv: https://arxiv.org/abs/2403.13335 · HTML: https://mlsanity.com/paper/2403.13335/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 85 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [FNet: Mixing Tokens with Fourier Transforms](https://mlsanity.com/paper/2105.03824/) (2021)
- [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020)
- [Visual In-Context Learning for Large Vision-Language Models](https://mlsanity.com/paper/2402.11574/) (2024)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [ChatGPT: The End of Online Exam Integrity?](https://mlsanity.com/paper/2212.09292/) (2022)
- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2402.10350/) (2024)
- [MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning](https://mlsanity.com/paper/2312.08636/) (2023)
- [SwitchTab: Switched Autoencoders Are Effective Tabular Learners](https://mlsanity.com/paper/2401.02013/) (2024)
- [Deepfake Text Detection: Limitations and Opportunities](https://mlsanity.com/paper/2210.09421/) (2022)
- [A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis](https://mlsanity.com/paper/2307.01981/) (2023)
- [Generative AI Text Classification using Ensemble LLM Approaches](https://mlsanity.com/paper/2309.07755/) (2023)
- [ReConTab: Regularized Contrastive Representation Learning for Tabular Data](https://mlsanity.com/paper/2310.18541/) (2023)

## Built on by (most cited first, 22 of 22)

- [BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval](https://mlsanity.com/paper/2403.15992/) (2024)
- [Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models](https://mlsanity.com/paper/2403.17359/) (2024)
- [Residual-based Language Models are Free Boosters for Biomedical Imaging](https://mlsanity.com/paper/2403.17343/) (2024)
- [MemoryMamba: Memory-Augmented State Space Model for Defect Recognition](https://mlsanity.com/paper/2405.03673/) (2024)
- [Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach](https://mlsanity.com/paper/2408.06634/) (2024)
- [MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale](https://mlsanity.com/paper/2404.12372/) (2024)
- [The New Agronomists: Language Models are Experts in Crop Management](https://mlsanity.com/paper/2403.19839/) (2024)
- [InsectMamba: Insect Pest Classification with State Space Model](https://mlsanity.com/paper/2404.03611/) (2024)
- [LLMExplainer: Large Language Model based Bayesian Inference for Graph Explanation Generation](https://mlsanity.com/paper/2407.15351/) (2024)
- [Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm](https://mlsanity.com/paper/2509.07287/) (2025)
- [Beyond Easy Wins: A Text Hardness-Aware Benchmark for LLM-generated Text Detection](https://mlsanity.com/paper/2507.15286/) (2025)
- [Deep Representation Learning for Multi-functional Degradation Modeling of Community-dwelling Aging Population](https://mlsanity.com/paper/2404.05613/) (2024)
- [Unmasking the Imposters: How Censorship and Domain Adaptation Affect the Detection of Machine-Generated Tweets](https://mlsanity.com/paper/2406.17967/) (2024)
- [ConspEmoLLM-v2: A robust and stable model to detect sentiment-transformed conspiracy theories](https://mlsanity.com/paper/2505.14917/) (2025)
- [Towards Low-Energy Adaptive Personalization for Resource-Constrained Devices](https://mlsanity.com/paper/2403.15905/) (2024)
- [Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition](https://mlsanity.com/paper/2405.00307/) (2024)
- [CROPS: A Deployable Crop Management System Over All Possible State Availabilities](https://mlsanity.com/paper/2411.06034/) (2024)
- [Domain Gating Ensemble Networks for AI-Generated Text Detection](https://mlsanity.com/paper/2505.13855/) (2025)
- [LENS: Learning Ensemble Confidence from Neural States for Multi-LLM Answer Integration](https://mlsanity.com/paper/2507.23167/) (2025)
- [Self-Error Adjustment: Theory and Practice of Balancing Individual Performance and Diversity in Ensemble Learning](https://mlsanity.com/paper/2508.04948/) (2025)
- [SENTRA: Selected-Next-Token Transformer for LLM Text Detection](https://mlsanity.com/paper/2509.12385/) (2025)
- [Detecting AI-Generated Paraphrases in Bengali: A Comparative Study of Zero-Shot and Fine-Tuned Transformers](https://mlsanity.com/paper/2512.21709/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have reached human-like proficiency in generating diverse textual content, underscoring the necessity for effective fake text detection to avoid potential risks such as fake news in social media. Previous research has mostly tested single models on in-distribution datasets, limiting our understanding of how these models perform on different types of data for LLM-generated text detection task. We researched this by testing five specialized transformer-based models on both in-distribution and out-of-distribution datasets to better assess their performance and generalizability. Our results revealed that single transformer-based classifiers achieved decent performance on in-distribution dataset but limited generalization ability on out-of-distribution dataset. To improve it, we combined the individual classifiers models using adaptive ensemble algorithms, which improved the average accuracy significantly from 91.8% to 99.2% on an in-distribution test set and from 62.9% to 72.5% on an out-of-distribution test set. The results indicate the effectiveness, good generalization ability, and great potential of adaptive ensemble algorithms in LLM-generated text detection.