# QUDsim: Quantifying Discourse Similarities in LLM-Generated Text

arXiv: https://arxiv.org/abs/2504.09373 · HTML: https://mlsanity.com/paper/2504.09373/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 12 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [Are Large Language Models Capable of Generating Human-Level Narratives?](https://mlsanity.com/paper/2407.13248/) (2024)
- [Putting GPT-3's Creativity to the (Alternative Uses) Test](https://mlsanity.com/paper/2206.08932/) (2022)
- [Echoes in AI: Quantifying lack of plot diversity in LLM outputs](https://mlsanity.com/paper/2501.00273/) (2024)
- [Detection and Measurement of Syntactic Templates in Generated Text](https://mlsanity.com/paper/2407.00211/) (2024)
- [AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text](https://mlsanity.com/paper/2410.04265/) (2024)
- [We're Different, We're the Same: Creative Homogeneity Across LLMs](https://mlsanity.com/paper/2501.19361/) (2025)
- [Controllable Open-ended Question Generation with A New Question Type Ontology](https://mlsanity.com/paper/2107.00152/) (2021)
- [Suri: Multi-constraint Instruction Following for Long-form Text Generation](https://mlsanity.com/paper/2406.19371/) (2024)
- [Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections](https://mlsanity.com/paper/2111.00701/) (2021)
- [QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsing](https://mlsanity.com/paper/2310.14520/) (2023)
- [SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similarity](https://mlsanity.com/paper/2401.17072/) (2024)
- [Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs](https://mlsanity.com/paper/2402.10586/) (2024)
- [Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs](https://mlsanity.com/paper/2408.01417/) (2024)
- [Discourse Analysis via Questions and Answers: Parsing Dependency Structures of Questions Under Discussion](https://mlsanity.com/paper/2210.05905/) (2022)
- [Using Natural Language Explanations to Rescale Human Judgments](https://mlsanity.com/paper/2305.14770/) (2023)
- [Description-Based Text Similarity](https://mlsanity.com/paper/2305.12517/) (2023)
- [Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherence](https://mlsanity.com/paper/2402.10175/) (2024)
- [Characterising the Creative Process in Humans and Large Language Models](https://mlsanity.com/paper/2405.00899/) (2024)
- [The Paradox of Stochasticity: Limited Creativity and Computational Decoupling in Temperature-Varied LLM Outputs of Structured Fictional Data](https://mlsanity.com/paper/2502.08515/) (2025)
- [Detecting Narrative Elements in Informational Text](https://mlsanity.com/paper/2210.03028/) (2022)

## Built on by (most cited first, 13 of 13)

- [StoryScope: Investigating idiosyncrasies in AI fiction](https://mlsanity.com/paper/2604.03136/) (2026)
- [Measuring AI "Slop" in Text](https://mlsanity.com/paper/2509.19163/) (2025)
- [Discursive Circuits: How Do Language Models Understand Discourse Relations?](https://mlsanity.com/paper/2510.11210/) (2025)
- [AI generates well-liked but templatic empathic responses](https://mlsanity.com/paper/2604.08479/) (2026)
- [Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence](https://mlsanity.com/paper/2603.25537/) (2026)
- [Discourse Diversity in Multi-Turn Empathic Dialogue](https://mlsanity.com/paper/2604.11742/) (2026)
- [Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles](https://mlsanity.com/paper/2512.20780/) (2025)
- [Which course? Discourse! Teaching Discourse and Generation in the Era of LLMs](https://mlsanity.com/paper/2602.02878/) (2026)
- [DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering](https://mlsanity.com/paper/2604.15140/) (2026)
- [Multimodal QUD: Inquisitive Questions from Scientific Figures](https://mlsanity.com/paper/2604.23733/) (2026)
- [GENIE: A Fine-Grained Measure for Novelty](https://mlsanity.com/paper/2606.12790/) (2026)
- [Characterizing Cultural Localization in AI-Generated Stories](https://mlsanity.com/paper/2606.14626/) (2026)
- [CASPER in the Machine: Insights into Character Variety in LLM-Generated Stories](https://mlsanity.com/paper/2606.22454/) (2026)

## Abstract (arXiv)

As large language models become increasingly capable at various writing tasks, their weakness at generating unique and creative content becomes a major liability. Although LLMs have the ability to generate text covering diverse topics, there is an overall sense of repetitiveness across texts that we aim to formalize and quantify via a similarity metric. The familiarity between documents arises from the persistence of underlying discourse structures. However, existing similarity metrics dependent on lexical overlap and syntactic patterns largely capture textitcontent overlap, thus making them unsuitable for detecting textitstructural similarities. We introduce an abstraction based on linguistic theories in Questions Under Discussion (QUD) and question semantics to help quantify differences in discourse progression. We then use this framework to build QUDsim, a similarity metric that can detect discursive parallels between documents. Using QUDsim, we find that LLMs often reuse discourse structures (more so than humans) across samples, even when content differs. Furthermore, LLMs are not only repetitive and structurally uniform, but are also divergent from human authors in the types of structures they use.