# Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions

arXiv: https://arxiv.org/abs/1510.00831 · HTML: https://mlsanity.com/paper/1510.00831/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2015.

## Built on by (most cited first, 12 of 12)

- [I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts](https://mlsanity.com/paper/2505.19190/) (2025)
- [Multimodal Fusion Interactions: A Study of Human and Automatic Quantification](https://mlsanity.com/paper/2306.04125/) (2023)
- [A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions](https://mlsanity.com/paper/2306.02149/) (2023)
- [Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition](https://mlsanity.com/paper/2606.00959/) (2026)
- [Computationally Efficient Approximations for Matrix-based Renyi's Entropy](https://mlsanity.com/paper/2112.13720/) (2021)
- [Partial information decomposition: redundancy as information bottleneck](https://mlsanity.com/paper/2405.07665/) (2024)
- [What should a neuron aim for? Designing local objective functions based on information theory](https://mlsanity.com/paper/2412.02482/) (2024)
- [Dissipation alters modes of information encoding in small quantum reservoirs near criticality](https://mlsanity.com/paper/2412.18290/) (2024)
- [Broadcast Channel Cooperative Gain: An Operational Interpretation of Partial Information Decomposition](https://mlsanity.com/paper/2502.10878/) (2025)
- [Progressive Interpretation Synthesis: Interpreting Task Solving by Quantifying Previously Used and Unused Information](https://mlsanity.com/paper/2101.02879/) (2021)
- [Multimodal Representation-disentangled Information Bottleneck for Multimodal Recommendation](https://mlsanity.com/paper/2509.20225/) (2025)
- [Interpretable Information-Decomposed Brain Graph Learning for fMRI-based Disease Diagnosis](https://mlsanity.com/paper/2608.20380/) (2026)

## Abstract (arXiv)

In many neural systems anatomical motifs are present repeatedly, but despite their structural similarity they can serve very different tasks. A prime example for such a motif is the canonical microcircuit of six-layered neo-cortex, which is repeated across cortical areas, and is involved in a number of different tasks (e.g.sensory, cognitive, or motor tasks). This observation has spawned interest in finding a common underlying principle, a 'goal function', of information processing implemented in this structure. By definition such a goal function, if universal, cannot be cast in processing-domain specific language (e.g. 'edge filtering', 'working memory'). Thus, to formulate such a principle, we have to use a domain-independent framework. Information theory offers such a framework. However, while the classical framework of information theory focuses on the relation between one input and one output (Shannon's mutual information), we argue that neural information processing crucially depends on the combination of textitmultiple inputs to create the output of a processor. To account for this, we use a very recent extension of Shannon Information theory, called partial information decomposition (PID). PID allows to quantify the information that several inputs provide individually (unique information), redundantly (shared information) or only jointly (synergistic information) about the output. First, we review the framework of PID. Then we apply it to reevaluate and analyze several earlier proposals of information theoretic neural goal functions (predictive coding, infomax, coherent infomax, efficient coding). We find that PID allows to compare these goal functions in a common framework, and also provides a versatile approach to design new goal functions from first principles. Building on this, we design and analyze a novel goal function, called 'coding with synergy'. [...]