# Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text

arXiv: https://arxiv.org/abs/2307.11380 · HTML: https://mlsanity.com/paper/2307.11380/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text](https://mlsanity.com/paper/2301.13852/) (2023)
- [CSL: A Large-scale Chinese Scientific Literature Dataset](https://mlsanity.com/paper/2209.05034/) (2022)

## Built on by (most cited first, 15 of 15)

- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Towards Codable Watermarking for Injecting Multi-bits Information to LLMs](https://mlsanity.com/paper/2307.15992/) (2023)
- [Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles](https://mlsanity.com/paper/2502.18968/) (2025)
- [PlatoLM: Teaching LLMs in Multi-Round Dialogue via a User Simulator](https://mlsanity.com/paper/2308.11534/) (2023)
- [A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization](https://mlsanity.com/paper/2403.01152/) (2024)
- [Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing](https://mlsanity.com/paper/2502.15666/) (2025)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs](https://mlsanity.com/paper/2402.10586/) (2024)
- [Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement](https://mlsanity.com/paper/2410.14259/) (2024)
- [Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text](https://mlsanity.com/paper/2405.12689/) (2024)
- [MegaFake: A Theory-Driven Dataset of Fake News Generated by Large Language Models](https://mlsanity.com/paper/2408.11871/) (2024)
- [Towards Reliable Detection of LLM-Generated Texts: A Comprehensive Evaluation Framework with CUDRT](https://mlsanity.com/paper/2406.09056/) (2024)
- [Beyond the Final Actor: Modeling the Dual Roles of Creator and Editor for Fine-Grained LLM-Generated Text Detection](https://mlsanity.com/paper/2604.04932/) (2026)
- [IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detector](https://mlsanity.com/paper/2502.15902/) (2025)

## Abstract (arXiv)

The remarkable capabilities of large-scale language models, such as ChatGPT, in text generation have impressed readers and spurred researchers to devise detectors to mitigate potential risks, including misinformation, phishing, and academic dishonesty. Despite this, most previous studies have been predominantly geared towards creating detectors that differentiate between purely ChatGPT-generated texts and human-authored texts. This approach, however, fails to work on discerning texts generated through human-machine collaboration, such as ChatGPT-polished texts. Addressing this gap, we introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts), facilitating the construction of more robust detectors. It diverges from extant corpora by comprising pairs of human-written and ChatGPT-polished abstracts instead of purely ChatGPT-generated texts. Additionally, we propose the "Polish Ratio" method, an innovative measure of the degree of modification made by ChatGPT compared to the original human-written text. It provides a mechanism to measure the degree of ChatGPT influence in the resulting text. Our experimental results show our proposed model has better robustness on the HPPT dataset and two existing datasets (HC3 and CDB). Furthermore, the "Polish Ratio" we proposed offers a more comprehensive explanation by quantifying the degree of ChatGPT involvement.