# Traceability Transformed: Generating more Accurate Links with Pre-Trained BERT Models

arXiv: https://arxiv.org/abs/2102.04411 · HTML: https://mlsanity.com/paper/2102.04411/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2021.

## Built on by (most cited first, 20 of 20)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning](https://mlsanity.com/paper/2307.13770/) (2023)
- [Watermarking Techniques for Large Language Models: A Survey](https://mlsanity.com/paper/2409.00089/) (2024)
- [Learning to Reduce False Positives in Analytic Bug Detectors](https://mlsanity.com/paper/2203.09907/) (2022)
- [Automating Code-Related Tasks Through Transformers: The Impact of Pre-training](https://mlsanity.com/paper/2302.04048/) (2023)
- [Improving Code Autocompletion with Transfer Learning](https://mlsanity.com/paper/2105.05991/) (2021)
- [6GSoft: Software for Edge-to-Cloud Continuum](https://mlsanity.com/paper/2407.05963/) (2024)
- [Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes](https://mlsanity.com/paper/2502.09690/) (2025)
- [Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement](https://mlsanity.com/paper/2212.05970/) (2022)
- [CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back](https://mlsanity.com/paper/2302.03924/) (2023)
- [Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes](https://mlsanity.com/paper/2409.06416/) (2024)
- [LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery](https://mlsanity.com/paper/2508.12232/) (2025)
- [An Empirical Study on Data Leakage and Generalizability of Link Prediction Models for Issues and Commits](https://mlsanity.com/paper/2211.00381/) (2022)
- [Applying Large Language Models to Issue Classification: Revisiting with Extended Data and New Models](https://mlsanity.com/paper/2506.00128/) (2025)
- [Model-Driven Discipline for Multi-Agent LLMs: Requirement-to-Verification Generation of Traceable System Models](https://mlsanity.com/paper/2607.16708/) (2026)
- [TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development](https://mlsanity.com/paper/2607.18886/) (2026)
- [Lost in Transmission: An Information-Theoretic Account of Unsupervised Software Traceability](https://mlsanity.com/paper/2412.04704/) (2024)
- [Natural Language-Programming Language Software Traceability Link Recovery Needs More than Textual Similarity](https://mlsanity.com/paper/2509.05585/) (2025)
- [Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Management](https://mlsanity.com/paper/2606.17203/) (2026)
- [AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation](https://mlsanity.com/paper/2607.22898/) (2026)

## Abstract (arXiv)

Software traceability establishes and leverages associations between diverse development artifacts. Researchers have proposed the use of deep learning trace models to link natural language artifacts, such as requirements and issue descriptions, to source code; however, their effectiveness has been restricted by availability of labeled data and efficiency at runtime. In this study, we propose a novel framework called Trace BERT (T-BERT) to generate trace links between source code and natural language artifacts. To address data sparsity, we leverage a three-step training strategy to enable trace models to transfer knowledge from a closely related Software Engineering challenge, which has a rich dataset, to produce trace links with much higher accuracy than has previously been achieved. We then apply the T-BERT framework to recover links between issues and commits in Open Source Projects. We comparatively evaluated accuracy and efficiency of three BERT architectures. Results show that a Single-BERT architecture generated the most accurate links, while a Siamese-BERT architecture produced comparable results with significantly less execution time. Furthermore, by learning and transferring knowledge, all three models in the framework outperform classical IR trace models. On the three evaluated real-word OSS projects, the best T-BERT stably outperformed the VSM model with average improvements of 60.31% measured using Mean Average Precision (MAP). RNN severely underperformed on these projects due to insufficient training data, while T-BERT overcame this problem by using pretrained language models and transfer learning.