# Applying Information Theory to Software Evolution

arXiv: https://arxiv.org/abs/2303.13729 · HTML: https://mlsanity.com/paper/2303.13729/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2023.

## Built on by (most cited first, 1 of 1)

- [Measuring LLM Code Generation Stability via Structural Entropy](https://mlsanity.com/paper/2508.14288/) (2025)

## Abstract (arXiv)

Although information theory has found success in disciplines, the literature on its applications to software evolution is limit. We are still missing artifacts that leverage the data and tooling available to measure how the information content of a project can be a proxy for its complexity. In this work, we explore two definitions of entropy, one structural and one textual, and apply it to the historical progression of the commit history of 25 open source projects. We produce evidence that they generally are highly correlated. We also observed that they display weak and unstable correlations with other complexity metrics. Our preliminary investigation of outliers shows an unexpected high frequency of events where there is considerable change in the information content of the project, suggesting that such outliers may inform a definition of surprisal.