# Lost in Transmission: An Information-Theoretic Account of Unsupervised Software Traceability

arXiv: https://arxiv.org/abs/2412.04704 · HTML: https://mlsanity.com/paper/2412.04704/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code](https://mlsanity.com/paper/2003.07914/) (2020)
- [Semantically Enhanced Software Traceability Using Deep Learning Techniques](https://mlsanity.com/paper/1804.02438/) (2018)
- [Traceability Transformed: Generating more Accurate Links with Pre-Trained BERT Models](https://mlsanity.com/paper/2102.04411/) (2021)
- [EALink: An Efficient and Accurate Pre-trained Framework for Issue-Commit Link Recovery](https://mlsanity.com/paper/2308.10759/) (2023)
- [Improving the Effectiveness of Traceability Link Recovery using Hierarchical Bayesian Networks](https://mlsanity.com/paper/2005.09046/) (2020)
- [Learning to Identify Security-Related Issues Using Convolutional Neural Networks](https://mlsanity.com/paper/1908.00614/) (2019)
- [Leveraging Historical Associations between Requirements and Source Code to Identify Impacted Classes](https://mlsanity.com/paper/1808.06359/) (2018)
- [Decomposing information into copying versus transformation](https://mlsanity.com/paper/1903.10693/) (2019)
- [Enhancing Automated Software Traceability by Transfer Learning from Open-World Data](https://mlsanity.com/paper/2207.01084/) (2022)

## Abstract (arXiv)

Traceability remains a critical capability to ensure system reliability, maintainability, and compliance in modern software development. Although unsupervised Information Retrieval (IR) and Machine Learning (ML) techniques are widely adopted for automated trace link recovery, their effectiveness is often limited by the quality and structure of the underlying artifacts. In practice, these approaches assume that meaningful traceability signals are embedded in textual data, an assumption that rarely holds in industrial settings with sparse, inconsistent, or unbalanced documentation. Furthermore, conventional evaluation metrics (e.g., precision, recall, F1) can misrepresent performance when data characteristics are not explicitly considered. We introduce TraceXplainer, an information-theoretic framework for evaluating the reliability and limits of unsupervised traceability. Our approach leverages self-information and mutual information (MI) to quantify the informativeness and alignment of source and target artifacts. Through a comprehensive empirical analysis of industry datasets, we show that typical traceability corpora exhibit significant information imbalances, where the source code contains on average more information than the corresponding documentation. In addition, the observed levels of mutual information, loss, and noise reveal inherent constraints on the ability of unsupervised techniques to recover accurate trace links. These findings suggest that improving traceability in practice requires a shift to data-centric engineering, focusing on artifact quality, consistency, and information alignment; rather than solely advancing model sophistication (or complexity). Our results provide insights for practitioners to better assess traceability readiness and guide improvements in documentation and development workflows.