# Prompts Matter: Insights and Strategies for Prompt Engineering in Automated Software Traceability

arXiv: https://arxiv.org/abs/2308.00229 · HTML: https://mlsanity.com/paper/2308.00229/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2023.

## Built on by (most cited first, 15 of 15)

- [MARE: Multi-Agents Collaboration Framework for Requirements Engineering](https://mlsanity.com/paper/2405.03256/) (2024)
- [Model Generation with LLMs: From Requirements to UML Sequence Diagrams](https://mlsanity.com/paper/2404.06371/) (2024)
- [Analyzing Prompt Influence on Automated Method Generation: An Empirical Study with Copilot](https://mlsanity.com/paper/2402.08430/) (2024)
- [Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineering](https://mlsanity.com/paper/2411.15501/) (2024)
- [Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Code](https://mlsanity.com/paper/2504.13656/) (2025)
- [Benchmarking LLMs on the Semantic Overlap Summarization Task](https://mlsanity.com/paper/2402.17008/) (2024)
- [Augmenting the Author: Exploring the Potential of AI Collaboration in Academic Writing](https://mlsanity.com/paper/2404.16071/) (2024)
- [Can LLMs Generate User Stories and Assess Their Quality?](https://mlsanity.com/paper/2507.15157/) (2025)
- [LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery](https://mlsanity.com/paper/2508.12232/) (2025)
- [aLLoyM: A large language model for alloy phase diagram prediction](https://mlsanity.com/paper/2507.22558/) (2025)
- [TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development](https://mlsanity.com/paper/2607.18886/) (2026)
- [Leveraging Prompts in LLMs to Overcome Imbalances in Complex Educational Text Data](https://mlsanity.com/paper/2407.01551/) (2024)
- [Towards Energy-aware Requirements Dependency Classification: Knowledge-Graph vs. Vector-Retrieval Augmented Inference with SLMs](https://mlsanity.com/paper/2603.23954/) (2026)
- [Large Language Models for Analyzing Enterprise Architecture Debt in Unstructured Documentation](https://mlsanity.com/paper/2604.00046/) (2026)
- [Trust-Aware Multi-Agent Traceability: Confidence-Calibrated Knowledge Graphs for Consistent Software Artifact Management](https://mlsanity.com/paper/2606.17203/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have the potential to revolutionize automated traceability by overcoming the challenges faced by previous methods and introducing new possibilities. However, the optimal utilization of LLMs for automated traceability remains unclear. This paper explores the process of prompt engineering to extract link predictions from an LLM. We provide detailed insights into our approach for constructing effective prompts, offering our lessons learned. Additionally, we propose multiple strategies for leveraging LLMs to generate traceability links, improving upon previous zero-shot methods on the ranking of candidate links after prompt refinement. The primary objective of this paper is to inspire and assist future researchers and engineers by highlighting the process of constructing traceability prompts to effectively harness LLMs for advancing automatic traceability.