# Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions

arXiv: https://arxiv.org/abs/2410.02028 · HTML: https://mlsanity.com/paper/2410.02028/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method](https://mlsanity.com/paper/2305.13412/) (2023)
- [Understanding Iterative Revision from Human-Written Text](https://mlsanity.com/paper/2203.03802/) (2022)
- [Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review](https://mlsanity.com/paper/2204.10805/) (2022)
- [Merging Generated and Retrieved Knowledge for Open-Domain QA](https://mlsanity.com/paper/2310.14393/) (2023)
- [In-Context Learning for Text Classification with Many Labels](https://mlsanity.com/paper/2309.10954/) (2023)
- [Adversarial Domain Adaptation for Duplicate Question Detection](https://mlsanity.com/paper/1809.02255/) (2018)
- [arXivEdits: Understanding the Human Revision Process in Scientific Writing](https://mlsanity.com/paper/2210.15067/) (2022)
- [Exploring Zero and Few-shot Techniques for Intent Classification](https://mlsanity.com/paper/2305.07157/) (2023)
- [Position: Key Claims in LLM Research Have a Long Tail of Footnotes](https://mlsanity.com/paper/2308.07120/) (2023)
- [Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation](https://mlsanity.com/paper/2406.12114/) (2024)
- [Cross-domain Semantic Parsing via Paraphrasing](https://mlsanity.com/paper/1704.05974/) (2017)
- [Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision](https://mlsanity.com/paper/2406.00197/) (2024)
- [Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data](https://mlsanity.com/paper/2404.02422/) (2024)

## Built on by (most cited first, 11 of 11)

- [GenCLS++: Pushing the Boundaries of Generative Classification in LLMs Through Comprehensive SFT and RL Studies Across Diverse Datasets](https://mlsanity.com/paper/2504.19898/) (2025)
- [Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review](https://mlsanity.com/paper/2602.11173/) (2026)
- [Fair Text Classification via Transferable Representations](https://mlsanity.com/paper/2503.07691/) (2025)
- [Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction](https://mlsanity.com/paper/2510.00268/) (2025)
- [Expos'ia: Teaching and Assessment of Academic Writing Skills for Research Project Proposals and Peer Feedback](https://mlsanity.com/paper/2601.06536/) (2026)
- [MASS-RAG: Multi-Agent Synthesis Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.18509/) (2026)
- [IntentGrasp: A Comprehensive Benchmark for Intent Understanding](https://mlsanity.com/paper/2605.06832/) (2026)
- [Evaluating the Effectiveness of Linguistic Knowledge in Pretrained Language Models: A Case Study of Universal Dependencies](https://mlsanity.com/paper/2506.04887/) (2025)
- [Zero-shot reasoning for simulating scholarly peer-review](https://mlsanity.com/paper/2510.02027/) (2025)
- [Intention-Adaptive LLM Fine-Tuning for Text Revision Generation](https://mlsanity.com/paper/2602.00477/) (2026)
- [From Granular Revision Operations to Meaningful Revision Units: Evaluating LLMs for Revision Boundary Detection](https://mlsanity.com/paper/2609.33720/) (2026)

## Abstract (arXiv)

Classification is a core NLP task architecture with many potential applications. While large language models (LLMs) have brought substantial advancements in text generation, their potential for enhancing classification tasks remains underexplored. To address this gap, we propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task. Our extensive experiments and systematic comparisons with various training approaches and a representative selection of LLMs yield new insights into their application for EIC. We investigate the generalizability of these findings on five further classification tasks. To demonstrate the proposed methods and address the data shortage for empirical edit analysis, we use our best-performing EIC model to create Re3-Sci2.0, a new large-scale dataset of 1,780 scientific document revisions with over 94k labeled edits. The quality of the dataset is assessed through human evaluation. The new dataset enables an in-depth empirical study of human editing behavior in academic writing. We make our experimental framework, models and data publicly available.