# Information Theory: A Tutorial Introduction

arXiv: https://arxiv.org/abs/1802.05968 · HTML: https://mlsanity.com/paper/1802.05968/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2018.

## Built on by (most cited first, 7 of 7)

- [What Makes for Robust Multi-Modal Models in the Face of Missing Modalities?](https://mlsanity.com/paper/2310.06383/) (2023)
- [Strong Transitivity Relations and Graph Neural Networks](https://mlsanity.com/paper/2401.01384/) (2024)
- [The Rate-Distortion-Polysemanticity Tradeoff in SAEs](https://mlsanity.com/paper/2605.14694/) (2026)
- [NECA: Network-Embedded Deep Representation Learning for Categorical Data](https://mlsanity.com/paper/2205.12752/) (2022)
- [Finding Patterns in Visualized Data by Adding Redundant Visual Information](https://mlsanity.com/paper/2205.13856/) (2022)
- [Redundancy Analysis and Mitigation for Machine Learning-Based Process Monitoring of Additive Manufacturing](https://mlsanity.com/paper/2504.21317/) (2025)
- [Synergizing Reinforcement Learning and Genetic Algorithms for Neural Combinatorial Optimization](https://mlsanity.com/paper/2506.09404/) (2025)

## Abstract (arXiv)

Shannon's mathematical theory of communication defines fundamental limits on how much information can be transmitted between the different components of any man-made or biological system. This paper is an informal but rigorous introduction to the main ideas implicit in Shannon's theory. An annotated reading list is provided for further reading.