# Information Dynamics of Language Communication

arXiv: https://arxiv.org/abs/2606.30096 · HTML: https://mlsanity.com/paper/2606.30096/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 72 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?](https://mlsanity.com/paper/2212.12131/) (2022)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [ConvoKit: A Toolkit for the Analysis of Conversations](https://mlsanity.com/paper/2005.04246/) (2020)
- [Granger Causality: A Review and Recent Advances](https://mlsanity.com/paper/2105.02675/) (2021)
- [Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems](https://mlsanity.com/paper/1411.2832/) (2014)
- [Measuring multivariate redundant information with pointwise common change in surprisal](https://mlsanity.com/paper/1602.05063/) (2016)
- [A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directions](https://mlsanity.com/paper/2502.11095/) (2025)
- [A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions](https://mlsanity.com/paper/2306.02149/) (2023)
- [Modeling Interpersonal Linguistic Coordination in Conversations using Word Mover's Distance](https://mlsanity.com/paper/1904.06002/) (2019)
- [Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?](https://mlsanity.com/paper/2311.09325/) (2023)
- [Entropy-Lens: Uncovering Decision Strategies in LLMs](https://mlsanity.com/paper/2502.16570/) (2025)
- [A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning](https://mlsanity.com/paper/2601.06851/) (2026)
- [On the Entropy Calibration of Language Models](https://mlsanity.com/paper/2511.11966/) (2025)
- [Predicting Long-Term Citations from Short-Term Linguistic Influence](https://mlsanity.com/paper/2210.13628/) (2022)
- [Shannon invariants: A scalable approach to information decomposition](https://mlsanity.com/paper/2504.15779/) (2025)

## Abstract (arXiv)

Quantifying how meaning propagates through communicative exchanges remains underdeveloped in computational linguistics. Here we introduce an information-theoretic framework that quantifies the directed flow of semantic content between interlocutors and decomposes multi-source contributions into redundant, unique, and synergistic components. Our approach leverages large language models as probabilistic estimators of natural language to compute two measures: semantic transfer entropy (STE), which captures directed predictive influence between speakers, and semantic partial information decomposition (SPID), which resolves how multiple sources jointly shape a target's language. Across four experiments we show that the framework detects reduced information flow in cognitively rigid dialogue, captures the dominant role of persuaders in shaping discourse, distinguishes high- from low-quality psychotherapy by the directionality of therapist-client information exchange, and reveals synergistic premise contributions in argumentative essays. This framework opens new avenues for studying information dynamics in digital discourse, pedagogical interactions, clinical dialogues, and any domain in which the structure of linguistic exchange is of research relevance.