# Efficient Quantification of Multimodal Interaction at Sample Level

arXiv: https://arxiv.org/abs/2506.17248 · HTML: https://mlsanity.com/paper/2506.17248/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2025.

## Built on by (most cited first, 7 of 7)

- [To Align or Not to Align: Strategic Multimodal Representation Alignment for Optimal Performance](https://mlsanity.com/paper/2511.12121/) (2025)
- [How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoning](https://mlsanity.com/paper/2602.15580/) (2026)
- [Massively Multimodal Foundation Models: A Framework for Capturing Interactions with Specialized Mixture-of-Experts](https://mlsanity.com/paper/2509.25678/) (2025)
- [MIBench: Evaluating LMMs on Multimodal Interaction](https://mlsanity.com/paper/2603.13427/) (2026)
- [Self-Captioning Multimodal Interaction Tuning: Amplifying Exploitable Redundancies for Robust Vision Language Models](https://mlsanity.com/paper/2605.08145/) (2026)
- [Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion](https://mlsanity.com/paper/2610.07466/) (2026)
- [Are Multimodal Benchmarks Really Useful? Item-Level Multimodal Benchmark Diagnosis via Structure-Response Co-Calibration](https://mlsanity.com/paper/2610.14542/) (2026)

## Abstract (arXiv)

Interactions between modalities -- redundancy, uniqueness, and synergy -- collectively determine the composition of multimodal information. Understanding these interactions is crucial for analyzing information dynamics in multimodal systems, yet their accurate sample-level quantification presents significant theoretical and computational challenges. To address this, we introduce the Lightweight Sample-wise Multimodal Interaction (LSMI) estimator, rigorously grounded in pointwise information theory. We first develop a redundancy estimation framework, employing an appropriate pointwise information measure to quantify this most decomposable and measurable interaction. Building upon this, we propose a general interaction estimation method that employs efficient entropy estimation, specifically tailored for sample-wise estimation in continuous distributions. Extensive experiments on synthetic and real-world datasets validate LSMI's precision and efficiency. Crucially, our sample-wise approach reveals fine-grained sample- and category-level dynamics within multimodal data, enabling practical applications such as redundancy-informed sample partitioning, targeted knowledge distillation, and interaction-aware model ensembling. The code is available at https://github.com/GeWu-Lab/LSMI_Estimator.