# TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications

arXiv: https://arxiv.org/abs/2406.01768 · HTML: https://mlsanity.com/paper/2406.01768/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities](https://mlsanity.com/paper/2309.16739/) (2023)
- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [Large Language Models Empowered Autonomous Edge AI for Connected Intelligence](https://mlsanity.com/paper/2307.02779/) (2023)
- [Large Language Models for Telecom: Forthcoming Impact on the Industry](https://mlsanity.com/paper/2308.06013/) (2023)
- [WirelessLLM: Empowering Large Language Models Towards Wireless Intelligence](https://mlsanity.com/paper/2405.17053/) (2024)
- [Understanding Telecom Language Through Large Language Models](https://mlsanity.com/paper/2306.07933/) (2023)
- [Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6G](https://mlsanity.com/paper/2405.02336/) (2024)
- [Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunications](https://mlsanity.com/paper/2404.15939/) (2024)
- [SPEC5G: A Dataset for 5G Cellular Network Protocol Analysis](https://mlsanity.com/paper/2301.09201/) (2023)
- [The Power of Large Language Models for Wireless Communication System Development: A Case Study on FPGA Platforms](https://mlsanity.com/paper/2307.07319/) (2023)
- [AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems](https://mlsanity.com/paper/2311.05842/) (2023)
- [Telecom Language Models: Must They Be Large?](https://mlsanity.com/paper/2403.04666/) (2024)
- [Adapting Foundation Models for Information Synthesis of Wireless Communication Specifications](https://mlsanity.com/paper/2308.04033/) (2023)

## Built on by (most cited first, 27 of 27)

- [SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging](https://mlsanity.com/paper/2503.17239/) (2025)
- [ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networks](https://mlsanity.com/paper/2407.06245/) (2024)
- [AgentRAN: An Agentic AI Architecture for Autonomous Control of Open 6G Networks](https://mlsanity.com/paper/2508.17778/) (2025)
- [ORANSight-2.0: Foundational LLMs for O-RAN](https://mlsanity.com/paper/2503.05200/) (2025)
- [Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards](https://mlsanity.com/paper/2408.11775/) (2024)
- [TeleTables: A Benchmark for Large Language Models in Telecom Table Interpretation](https://mlsanity.com/paper/2601.04202/) (2025)
- [AI5GTest: AI-Driven Specification-Aware Automated Testing and Validation of 5G O-RAN Components](https://mlsanity.com/paper/2506.10111/) (2025)
- [Telco-DPR: A Hybrid Dataset for Evaluating Retrieval Models of 3GPP Technical Specifications](https://mlsanity.com/paper/2410.19790/) (2024)
- [Enhancing Large Language Models (LLMs) for Telecom using Dynamic Knowledge Graphs and Explainable Retrieval-Augmented Generation](https://mlsanity.com/paper/2602.17529/) (2026)
- [TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network](https://mlsanity.com/paper/2411.02617/) (2024)
- [Towards a Foundation Model for Communication Systems](https://mlsanity.com/paper/2505.14603/) (2025)
- [From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications](https://mlsanity.com/paper/2505.22311/) (2025)
- [TeleResilienceBench: Quantifying Resilience for LLM Reasoning in Telecommunications](https://mlsanity.com/paper/2605.09929/) (2026)
- [GENESIS: Harnessing AI Agents for Autonomous 6G RAN Synthesis, Research, and Testing](https://mlsanity.com/paper/2605.27360/) (2026)
- [Distilling Large Language Models for Network Active Queue Management](https://mlsanity.com/paper/2501.16734/) (2025)
- [Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.24245/) (2025)
- [How to Discover Knowledge for FutureG: Contextual RAG and LLM Prompting for O-RAN](https://mlsanity.com/paper/2601.02382/) (2025)
- [TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?](https://mlsanity.com/paper/2605.18025/) (2026)
- [TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks](https://mlsanity.com/paper/2609.25356/) (2026)
- [Rephrase and Contrast: Fine-Tuning Language Models for Enhanced Understanding of Communication and Computer Networks](https://mlsanity.com/paper/2409.19007/) (2024)
- [SafeCOMM: A Study on Safety Degradation in Fine-Tuned Telecom Large Language Models](https://mlsanity.com/paper/2506.00062/) (2025)
- [Concept-Level AI for Telecom: Moving Beyond Large Language Models](https://mlsanity.com/paper/2506.22359/) (2025)
- [DeepSpecs: Expert-Level Questions Answering in 5G](https://mlsanity.com/paper/2511.01305/) (2025)
- [MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications](https://mlsanity.com/paper/2511.13131/) (2025)
- [TelcoAI: Advancing 3GPP Technical Specification Search through Agentic Multi-Modal Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.16984/) (2025)
- [Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline](https://mlsanity.com/paper/2606.29014/) (2026)
- [AutoSec-RAN: Specification-Grounded LLM Fine-Tuning for Automated Security Analysis of O-RAN](https://mlsanity.com/paper/2610.15995/) (2026)

## Abstract (arXiv)

Understanding telecom standards involves sorting through numerous technical documents, such as those produced by the 3rd Generation Partnership Project (3GPP), which is time-consuming and labor-intensive. While large language models (LLMs) can assist with the extensive 3GPP knowledge base, an inclusive dataset is crucial for their effective pre-training and fine-tuning. In this paper, we introduce textitTSpec-LLM, an open-source comprehensive dataset covering all 3GPP documents from Release 8 to Release 19 (1999--2023). To evaluate its efficacy, we first select a representative sample of 3GPP documents, create corresponding technical questions, and assess the baseline performance of various LLMs. We then incorporate a retrieval-augmented generation (RAG) framework to enhance LLM capabilities by retrieving relevant context from the textitTSpec-LLM dataset. Our evaluation shows that using a naive-RAG framework on textitTSpec-LLM improves the accuracy of GPT-3.5, Gemini 1.0 Pro, and GPT-4 from 44%, 46%, and 51% to 71%, 75%, and 72%, respectively.