mlsanityData to 14 Oct 2026

Do LLMs write like humans? Variation in grammatical and rhetorical styles

arXiv 2410.16107 ↗First posted 21 Oct 2024cs.CLcited by 62

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are capable of writing grammatical text that follows instructions, answers questions, and solves problems. As they have advanced, it has become difficult to distinguish their output from human-written text. While past research has found some differences in surface features such as word choice and punctuation, and developed classifiers to detect LLM output, none has studied the rhetorical styles of LLMs. Using several variants of Llama 3 and GPT-4o, we construct two parallel corpora of human- and LLM-written texts from common prompts. Using Douglas Biber's set of lexical, grammatical, and rhetorical features, we identify systematic differences between LLMs and humans and between different LLMs. These differences persist when moving from smaller models to larger ones, and are larger for instruction-tuned models than base models. This observation of differences demonstrates that despite their advanced abilities, LLMs struggle to match human stylistic variation. Attention to more advanced linguistic features can hence detect patterns in their behavior not previously recognized.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 14 of its 52 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

14 of 52 references.

References · 14

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  3. 2015ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and Rcited by 172
  4. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  5. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 129
  6. 2023The Science of Detecting LLM-Generated Textscited by 128
  7. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 121
  8. 2023Deception Abilities Emerged in Large Language Modelscited by 101
  9. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 94
  10. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  11. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  12. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
2 more
  1. 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Biascited by 53
  2. 2024ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictorcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 20; 2026: 41.

62 papers.

Sort:

By area:

  1. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 20
  2. 2025Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Stylescited by 16
  3. 2025Comparing LLM-generated and human-authored news text using formal syntactic theorycited by 11
  4. 2025A Comparison of DeepSeek and Other LLMscited by 10
  5. 2026StoryScope: Investigating idiosyncrasies in AI fictioncited by 9
  6. 2025Benchmark of stylistic variation in LLM-generated textscited by 7
  7. 2025Linguistic Characteristics of AI-Generated Text: A Surveycited by 7
  8. 2026Post-training makes large language models less human-likecited by 5
  9. 2025Wikipedia in the Era of LLMs: Evolution and Riskscited by 4
  10. 2025Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplificationcited by 4
  11. 2025When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Textcited by 4
  12. 2026PeerPrism: Peer Evaluation Expertise vs Review-writing AIcited by 4
38 more
  1. 2025Evaluating Style-Personalized Text Generation: Challenges and Directionscited by 3
  2. 2025On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?cited by 3
  3. 2026Beyond the Final Actor: Modeling the Dual Roles of Creator and Editor for Fine-Grained LLM-Generated Text Detectioncited by 3
  4. 2026From Safety Risk to Design Principle: Peer-Preservation in Multi-Agent LLM Systems and Its Implications for Orchestrated Democratic Discourse Analysiscited by 3
  5. 2026AI generates well-liked but templatic empathic responsescited by 3
  6. 2026Rhetorical Questions in LLM Representations: A Linear Probing Studycited by 3
  7. 2026More Aligned, Less Diverse? Analyzing the Grammar and Lexicon of Two Generations of LLMscited by 3
  8. 2024ExpressivityBench: Can LLMs Communicate Implicitly?cited by 2
  9. 2025Beyond checkmate: exploring the creative chokepoints in AI textcited by 2
  10. 2025References Matter: Investigating the Impact of Reference Set Variation on Summarization Evaluationcited by 2
  11. 2026Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classificationcited by 2
  12. 2026The Statistical Signature of LLMscited by 2
  13. 2025Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approachcited by 1
  14. 2025V-VAE: A Variational Auto Encoding Framework Towards Fine-Grained Control over Human-Like Chatcited by 1
  15. 2025Do LLMs produce texts with "human-like" lexical diversity?cited by 1
  16. 2025Attribution Quality in AI-Generated Content:Benchmarking Style Embeddings and LLM Judgescited by 1
  17. 2025HLPD: Aligning LLMs to Human Language Preference for Machine-Revised Text Detectioncited by 1
  18. 2025AudAgent: Automated Auditing of Privacy Policy Compliance in AI Agentscited by 1
  19. 2026PersonaMail: Learning and Adapting Personal Communication Preferences for Context-Aware Email Writingcited by 1
  20. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  21. 2026Interpretable Stylistic Variation in Human and LLM Writing Across Genres, Models, and Decoding Strategiescited by 1
  22. 2026Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipelinecited by 1
  23. 2026Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysiscited by 1
  24. 2026Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decodingcited by 1
  25. 2025Computational Phenomenology of Borderline Personality Disorder: A Comparative Evaluation of LLM-Simulated Expert Personas and Human Clinical Expertscited by 0
  26. 2025Large Language Models Preserve Semantic Isotopies in Story Continuationscited by 0
  27. 2026Do Large Language Models Adapt to Language Variation across Socioeconomic Status?cited by 0
  28. 2026The Astonishing Ability of Large Language Models to Parse Jabberwockified Languagecited by 0
  29. 2026DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answeringcited by 0
  30. 2026Beyond One Output: Visualizing and Comparing Distributions of Language Model Generationscited by 0
  31. 2026Voice Under Revision: Large Language Models and the Normalization of Personal Narrativecited by 0
  32. 2026The textitSilicon Society Cookbook: Design Space of LLM-based Social Simulationscited by 0
  33. 2026Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLMcited by 0
  34. 2026Beyond Transcripts: Iterative Peer-Editing with Audio Unlocks High-Quality Human Summaries of Conversational Speechcited by 0
  35. 2026What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experiencecited by 0
  36. 2026How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Frameworkcited by 0
  37. 2026Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learningcited by 0
  38. 2026Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maximscited by 0

50 most cited of 62 All 62 (JSON)