# "I know myself better, but not really greatly": How Well Can LLMs Detect and Explain LLM-Generated Texts?

arXiv: https://arxiv.org/abs/2502.12743 · HTML: https://mlsanity.com/paper/2502.12743/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 10 of 10)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?](https://mlsanity.com/paper/2401.05952/) (2024)
- [M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2402.11175/) (2024)
- [A Survey on Large Language Models with some Insights on their Capabilities and Limitations](https://mlsanity.com/paper/2501.04040/) (2025)
- [Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?](https://mlsanity.com/paper/2308.01284/) (2023)
- [Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack](https://mlsanity.com/paper/2404.01907/) (2024)
- [Detecting Machine-Generated Texts: Not Just "AI vs Humans" and Explainability is Complicated](https://mlsanity.com/paper/2406.18259/) (2024)

## Built on by (most cited first, 1 of 1)

- [Using LLMs to Detect LLM-Generated Texts: A Cross-Generation Analysis](https://mlsanity.com/paper/2609.34691/) (2026)

## Abstract (arXiv)

Distinguishing between human- and LLM-generated texts is crucial given the risks associated with misuse of LLMs. This paper investigates detection and explanation capabilities of current LLMs across two settings: binary (human vs. LLM-generated) and ternary classification (including an ``undecided'' class). We evaluate 6 close- and open-source LLMs of varying sizes and find that self-detection (LLMs identifying their own outputs) consistently outperforms cross-detection (identifying outputs from other LLMs), though both remain suboptimal. Introducing a ternary classification framework improves both detection accuracy and explanation quality across all models. Through comprehensive quantitative and qualitative analyses using our human-annotated dataset, we identify key explanation failures, primarily reliance on inaccurate features, hallucinations, and flawed reasoning. Our findings underscore the limitations of current LLMs in self-detection and self-explanation, highlighting the need for further research to address overfitting and enhance generalizability.