# Consistency of Responses and Continuations Generated by Large Language Models on Social Media

arXiv: https://arxiv.org/abs/2501.08102 · HTML: https://mlsanity.com/paper/2501.08102/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT](https://mlsanity.com/paper/2303.04226/) (2023)
- [A Survey of Evaluation Metrics Used for NLG Systems](https://mlsanity.com/paper/2008.12009/) (2020)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Achieving Human Parity on Automatic Chinese to English News Translation](https://mlsanity.com/paper/1803.05567/) (2018)
- [Evolution of Semantic Similarity -- A Survey](https://mlsanity.com/paper/2004.13820/) (2020)
- [A Complete Survey on LLM-based AI Chatbots](https://mlsanity.com/paper/2406.16937/) (2024)
- [Contrasting Linguistic Patterns in Human and LLM-Generated News Text](https://mlsanity.com/paper/2308.09067/) (2023)
- [HelloBench: Evaluating Long Text Generation Capabilities of Large Language Models](https://mlsanity.com/paper/2409.16191/) (2024)
- [On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation](https://mlsanity.com/paper/2005.01196/) (2020)
- [What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection](https://mlsanity.com/paper/2402.00371/) (2024)
- [Measuring Emotional Contagion in Social Media](https://mlsanity.com/paper/1506.06021/) (2015)
- [Context-Aware LLM Translation System Using Conversation Summarization and Dialogue History](https://mlsanity.com/paper/2410.16775/) (2024)
- [Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicity](https://mlsanity.com/paper/2407.05977/) (2024)

## Built on by (most cited first, 7 of 7)

- [From Personas to Talks: Revisiting the Impact of Personas on LLM-Synthesized Emotional Support Conversations](https://mlsanity.com/paper/2502.11451/) (2025)
- [Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMs](https://mlsanity.com/paper/2603.10011/) (2026)
- [A Dynamic Fusion Model for Consistent Crisis Response](https://mlsanity.com/paper/2509.01053/) (2025)
- [Fine-grained Verbal Attack Detection via a Hierarchical Divide-and-Conquer Framework](https://mlsanity.com/paper/2601.06907/) (2026)
- [LLMs Should Incorporate Explicit Mechanisms for Human Empathy](https://mlsanity.com/paper/2604.10557/) (2026)
- [DynamicPO: Dynamic Preference Optimization for Recommendation](https://mlsanity.com/paper/2605.00327/) (2026)
- [Towards Emotion Consistency Analysis of Large Language Models in Emotional Conversational Contexts](https://mlsanity.com/paper/2605.06476/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable capabilities in text generation, yet their emotional consistency and semantic coherence in social media contexts remain insufficiently understood. This study investigates how LLMs handle emotional content and maintain semantic relationships through continuation and response tasks using three open-source models: Gemma, Llama3 and Llama3.3 and one commercial Model:Claude. By analyzing climate change discussions from Twitter and Reddit, we examine emotional transitions, intensity patterns, and semantic consistency between human-authored and LLM-generated content. Our findings reveal that while both models maintain high semantic coherence, they exhibit distinct emotional patterns: these models show a strong tendency to moderate negative emotions. When the input text carries negative emotions such as anger, disgust, fear, or sadness, LLM tends to generate content with more neutral emotions, or even convert them into positive emotions such as joy or surprise. At the same time, we compared the LLM-generated content with human-authored content. The four models systematically generated responses with reduced emotional intensity and showed a preference for neutral rational emotions in the response task. In addition, these models all maintained a high semantic similarity with the original text, although their performance in the continuation task and the response task was different. These findings provide deep insights into the emotion and semantic processing capabilities of LLM, which are of great significance for its deployment in social media environments and human-computer interaction design.