# Predictive information in a sensory population

arXiv: https://arxiv.org/abs/1307.0225 · HTML: https://mlsanity.com/paper/1307.0225/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2013.

## Built on by (most cited first, 22 of 22)

- [To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review](https://mlsanity.com/paper/2304.09355/) (2023)
- [Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language](https://mlsanity.com/paper/2311.04930/) (2023)
- [Self-supervised video pretraining yields robust and more human-aligned visual representations](https://mlsanity.com/paper/2210.06433/) (2022)
- [Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization](https://mlsanity.com/paper/2107.01131/) (2021)
- [Recurrent Video Masked Autoencoders](https://mlsanity.com/paper/2512.13684/) (2025)
- [Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimality](https://mlsanity.com/paper/2208.03848/) (2022)
- [Learning predictable and robust neural representations by straightening image sequences](https://mlsanity.com/paper/2411.01777/) (2024)
- [Context Structure Reshapes the Representational Geometry of Language Models](https://mlsanity.com/paper/2601.22364/) (2026)
- [On the Effectiveness of Hybrid Mutual Information Estimation](https://mlsanity.com/paper/2306.00608/) (2023)
- [Perturbation Theory for the Information Bottleneck](https://mlsanity.com/paper/2105.13977/) (2021)
- [A polar prediction model for learning to represent visual transformations](https://mlsanity.com/paper/2303.03432/) (2023)
- [Linguistic Structure from a Bottleneck on Sequential Information Processing](https://mlsanity.com/paper/2405.12109/) (2024)
- [Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck](https://mlsanity.com/paper/2311.04896/) (2023)
- [Partial information decomposition: redundancy as information bottleneck](https://mlsanity.com/paper/2405.07665/) (2024)
- [Generalized Information Bottleneck for Gaussian Variables](https://mlsanity.com/paper/2303.17762/) (2023)
- [The Neuron as a Direct Data-Driven Controller](https://mlsanity.com/paper/2401.01489/) (2024)
- [Accurate Estimation of Mutual Information in High Dimensional Data](https://mlsanity.com/paper/2506.00330/) (2025)
- [Improving Numerical Stability of Normalized Mutual Information Estimator on High Dimensions](https://mlsanity.com/paper/2410.07642/) (2024)
- [Automated mapping of virtual environments with visual predictive coding](https://mlsanity.com/paper/2308.10913/) (2023)
- [Graceful forgetting: Memory as a process](https://mlsanity.com/paper/2502.11105/) (2025)
- [Semantic representations emerge in biologically inspired ensembles of cross-supervising neural networks](https://mlsanity.com/paper/2510.14486/) (2025)
- [Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks](https://mlsanity.com/paper/2609.04134/) (2026)

## Abstract (arXiv)

Guiding behavior requires the brain to make predictions about future sensory inputs. Here we show that efficient predictive computation starts at the earliest stages of the visual system. We estimate how much information groups of retinal ganglion cells carry about the future state of their visual inputs, and show that every cell we can observe participates in a group of cells for which this predictive information is close to the physical limit set by the statistical structure of the inputs themselves. Groups of cells in the retina also carry information about the future state of their own activity, and we show that this information can be compressed further and encoded by downstream predictor neurons, which then exhibit interesting feature selectivity. Efficient representation of predictive information is a candidate principle that can be applied at each stage of neural computation.