mlsanityData to 7 Oct 2026

Markovian Generation Chains in Large Language Models

arXiv 2603.11228 ↗First posted 11 Mar 2026cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous output as input, without including any prior memory. In iterative rephrasing and round-trip translation experiments, the output either converges to a small recurrent set or continues to produce novel sentences over a finite horizon. Through sentence-level Markov chain modeling and analysis of simulated data, we show that iterative process can either increase or reduce sentence diversity depending on factors such as the temperature parameter and the initial input sentence. These results offer valuable insights into the dynamics of iterative LLM inference and their implications for multi-agent LLM systems.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 11 Mar 2026.

Builds on

36 of 50 references.

References · 36

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2023Mistral 7Bcited by 3,065
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2018Hierarchical Neural Story Generationcited by 1,182
  7. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  8. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  9. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  10. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  11. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  12. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 143
24 more
  1. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
  2. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
  3. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  4. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 75
  5. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
  6. 2025Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Modelscited by 58
  7. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 57
  8. 2023Machine Culturecited by 55
  9. 2025Atom of Thoughts for Markov LLM Test-Time Scalingcited by 54
  10. 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 47
  11. 2024Benchmarking Linguistic Diversity of Large Language Modelscited by 37
  12. 2024AI and the Problem of Knowledge Collapsecited by 34
  13. 2024Large Language Models as Markov Chainscited by 33
  14. 2025Position: Model Collapse Does Not Mean What You Thinkcited by 32
  15. 2024Empirical evidence of Large Language Model's influence on human spoken communicationcited by 27
  16. 2024The Impact of Large Language Models in Academia: from Writing to Speakingcited by 18
  17. 2024When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settingscited by 17
  18. 2025Human-LLM Coevolution: Evidence from Academic Writingcited by 15
  19. 2025LLMs syntactically adapt their language use to their conversational partnercited by 15
  20. 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
  21. 2025LLM as a Broken Telephone: Iterative Generation Distorts Informationcited by 12
  22. 2025A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Biascited by 10
  23. 2025Do language models accommodate their users? A study of linguistic convergencecited by 7
  24. 2026Do LLMs Benefit From Their Own Words?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Interaction Creates Dynamical AI Behavior Absent in Isolationcited by 0
  2. 2026On the Indistinguishability of Human v/s AI Generated Textcited by 0
  3. 2026If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Caterpillar of Thoughts: The Optimal Test-Time Algorithm for Large Language Models
  2. 2026The Probabilistic Structure of Large Language Models
  3. 2024A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
  4. 2026Recipes for Steering and Scaling LLMs via Sampling
  5. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models
  6. 2026Diffusion Language Models: An Experimental Analysis
  7. 2025Convergence of Outputs When Two Large Language Models Interact in a Multi-Agentic Setup
  8. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions