# Long-form analogies generated by chatGPT lack human-like psycholinguistic properties

arXiv: https://arxiv.org/abs/2306.04537 · HTML: https://mlsanity.com/paper/2306.04537/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Hierarchical Clustering: Objective Functions and Algorithms](https://mlsanity.com/paper/1704.02147/) (2017)
- [Limits of Detecting Text Generated by Large-Scale Language Models](https://mlsanity.com/paper/2002.03438/) (2020)
- [TextDefense: Adversarial Text Detection based on Word Importance Entropy](https://mlsanity.com/paper/2302.05892/) (2023)
- [Predicting Early Indicators of Cognitive Decline from Verbal Utterances](https://mlsanity.com/paper/2012.02029/) (2020)

## Built on by (most cited first, 8 of 8)

- [BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data](https://mlsanity.com/paper/2410.16491/) (2024)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [Linguistic Characteristics of AI-Generated Text: A Survey](https://mlsanity.com/paper/2510.05136/) (2025)
- [Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm](https://mlsanity.com/paper/2509.07287/) (2025)
- [V-VAE: A Variational Auto Encoding Framework Towards Fine-Grained Control over Human-Like Chat](https://mlsanity.com/paper/2506.01524/) (2025)
- [Expanding the Set of Pragmatic Considerations in Conversational AI](https://mlsanity.com/paper/2310.18435/) (2023)
- [How Do LLM-Generated Texts Impact Term-Based Retrieval Models?](https://mlsanity.com/paper/2508.17715/) (2025)
- [Large Language Models Preserve Semantic Isotopies in Story Continuations](https://mlsanity.com/paper/2510.04400/) (2025)

## Abstract (arXiv)

Psycholinguistic analyses provide a means of evaluating large language model (LLM) output and making systematic comparisons to human-generated text. These methods can be used to characterize the psycholinguistic properties of LLM output and illustrate areas where LLMs fall short in comparison to human-generated text. In this work, we apply psycholinguistic methods to evaluate individual sentences from long-form analogies about biochemical concepts. We compare analogies generated by human subjects enrolled in introductory biochemistry courses to analogies generated by chatGPT. We perform a supervised classification analysis using 78 features extracted from Coh-metrix that analyze text cohesion, language, and readability (Graesser et. al., 2004). Results illustrate high performance for classifying student-generated and chatGPT-generated analogies. To evaluate which features contribute most to model performance, we use a hierarchical clustering approach. Results from this analysis illustrate several linguistic differences between the two sources.