mlsanityData to 14 Oct 2026

Information Dynamics of Language Communication

arXiv 2606.30096 ↗First posted 29 Jun 2026cs.CL cs.IT math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantifying how meaning propagates through communicative exchanges remains underdeveloped in computational linguistics. Here we introduce an information-theoretic framework that quantifies the directed flow of semantic content between interlocutors and decomposes multi-source contributions into redundant, unique, and synergistic components. Our approach leverages large language models as probabilistic estimators of natural language to compute two measures: semantic transfer entropy (STE), which captures directed predictive influence between speakers, and semantic partial information decomposition (SPID), which resolves how multiple sources jointly shape a target's language. Across four experiments we show that the framework detects reduced information flow in cognitively rigid dialogue, captures the dominant role of persuaders in shaping discourse, distinguishes high- from low-quality psychotherapy by the directionality of therapist-client information exchange, and reveals synergistic premise contributions in argumentative essays. This framework opens new avenues for studying information dynamics in digital discourse, pedagogical interactions, clinical dialogues, and any domain in which the structure of linguistic exchange is of research relevance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 72 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

24 of 72 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  7. 2024How to use and interpret activation patchingcited by 214
  8. 2019Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Goodcited by 209
  9. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
  10. 2016Parsing Argumentation Structures in Persuasive Essayscited by 101
  11. 2020Syntactic Structure from Deep Learningcited by 99
  12. 2020ConvoKit: A Toolkit for the Analysis of Conversationscited by 72
12 more
  1. 2021Granger Causality: A Review and Recent Advancescited by 63
  2. 2014Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systemscited by 16
  3. 2016Measuring multivariate redundant information with pointwise common change in surprisalcited by 13
  4. 2025A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directionscited by 12
  5. 2023A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functionscited by 8
  6. 2019Modeling Interpersonal Linguistic Coordination in Conversations using Word Mover's Distancecited by 7
  7. 2023Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?cited by 7
  8. 2025Entropy-Lens: Uncovering Decision Strategies in LLMscited by 6
  9. 2026A Brain-like Synergistic Core in LLMs Drives Behaviour and Learningcited by 6
  10. 2025On the Entropy Calibration of Language Modelscited by 4
  11. 2022Predicting Long-Term Citations from Short-Term Linguistic Influencecited by 3
  12. 2025Shannon invariants: A scalable approach to information decompositioncited by 2

Built on by

Not cited yet.