mlsanityData to 7 Oct 2026

Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information

arXiv 1902.11239 ↗First posted 28 Feb 2019cs.IT math.IT q-bio.NCcited by 30

Not on the map. Why?

Abstract (arXiv)

This article introduces a model-agnostic approach to study statistical synergy, a form of emergence in which patterns at large scales are not traceable from lower scales. Our framework leverages various multivariate extensions of Shannon's mutual information, and introduces the O-information as a metric capable of characterising synergy- and redundancy-dominated systems. We develop key analytical properties of the O-information, and study how it relates to other metrics of high-order interactions from the statistical mechanics and neuroscience literature. Finally, as a proof of concept, we use the proposed framework to explore the relevance of statistical synergy in Baroque music scores.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 4; 2024: 6; 2025: 6; 2026: 10.

30 papers.

Sort:

By area:

  1. 2024Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transitioncited by 16
  2. 2023Learning High-Order Relationships of Brain Regionscited by 10
  3. 2022Generative power of a protein language model trained on multiple sequence alignmentscited by 9
  4. 2023Information decomposition in complex systems via machine learningcited by 9
  5. 2025MvHo-IB: Multi-View Higher-Order Information Bottleneck for Brain Disorder Diagnosiscited by 8
  6. 2026The data geometry of masking diffusion: Certified-optimal schedules via unmasking growth complexitycited by 8
  7. 2022Synergistic information supports modality integration and flexible learning in neural networks solving multiple taskscited by 7
  8. 2025InfoBridge: Mutual Information estimation via Bridge Matchingcited by 7
  9. 2025A Generalized Information Bottleneck Theory of Deep Learningcited by 4
  10. 2025FMMI: Flow Matching Mutual Information Estimationcited by 4
  11. 2026A Statistical Framework for Auditing Behavioral Dependence and Induced Bias in LLM Judgescited by 4
  12. 2021Neural Dependency Coding inspired Multimodal Fusioncited by 3
18 more
  1. 2023Kernel-based Joint Independence Tests for Multivariate Stationary and Non-stationary Time Seriescited by 3
  2. 2023Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactionscited by 3
  3. 2024SOmegaI: Score-based O-INFORMATION Estimationcited by 3
  4. 2024Partial information decomposition: redundancy as information bottleneckcited by 3
  5. 2024Partial Information Decomposition for Data Interpretability and Feature Selectioncited by 3
  6. 2024Dissipation alters modes of information encoding in small quantum reservoirs near criticalitycited by 3
  7. 2026Towards Worst-Case Guarantees with Scale-Aware Interpretabilitycited by 3
  8. 2026Modeling Higher-Order Brain Interactions via a Multi-View Information Bottleneck Framework for fMRI-based Psychiatric Diagnosiscited by 3
  9. 2022Higher-order mutual information reveals synergistic sub-networks for multi-neuron importancecited by 1
  10. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  11. 2024Surveying the space of descriptions of a composite system with machine learningcited by 0
  12. 2025Cooperative effects in feature importance of individual patterns: application to air pollutants and Alzheimer diseasecited by 0
  13. 2025Information-theoretic signatures of causality in Bayesian networks and hypergraphscited by 0
  14. 2026HyperPotter: Spell the Charm of High-Order Interactions in Audio Deepfake Detectioncited by 0
  15. 2026Factorization Regret mediates compositional generalization in latent spacecited by 0
  16. 2026Dynamic Vine Copulas: Detecting and Quantifying Time-Varying Higher-Order Interactionscited by 0
  17. 2026Capacity and Redundancy Trade-offs in Multi-Task Learningcited by 0
  18. 2026ALICE: In-context, Zero-shot, Mutual Information Estimationcited by 0