# Using Large Language Models to Understand Telecom Standards

arXiv: https://arxiv.org/abs/2404.02929 · HTML: https://mlsanity.com/paper/2404.02929/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [Understanding Telecom Language Through Large Language Models](https://mlsanity.com/paper/2306.07933/) (2023)
- [Large Language Models for Networking: Applications, Enabling Techniques, and Challenges](https://mlsanity.com/paper/2311.17474/) (2023)
- [NetGPT: A Native-AI Network Architecture Beyond Provisioning Personalized Generative Services](https://mlsanity.com/paper/2307.06148/) (2023)
- [Making Network Configuration Human Friendly](https://mlsanity.com/paper/2309.06342/) (2023)
- [Observations on LLMs for Telecom Domain: Capabilities and Limitations](https://mlsanity.com/paper/2305.13102/) (2023)
- [AI-Generated Network Design: A Diffusion Model-based Learning Approach](https://mlsanity.com/paper/2303.13869/) (2023)

## Built on by (most cited first, 13 of 13)

- [Evaluation of RAG Metrics for Question Answering in the Telecom Domain](https://mlsanity.com/paper/2407.12873/) (2024)
- [Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards](https://mlsanity.com/paper/2408.11775/) (2024)
- [Mapping the Landscape of Generative AI in Network Monitoring and Management](https://mlsanity.com/paper/2502.08576/) (2025)
- [A Graph-based Approach for Multi-Modal Question Answering from Flowcharts in Telecom Documents](https://mlsanity.com/paper/2507.22938/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network](https://mlsanity.com/paper/2411.02617/) (2024)
- [A Survey on the Integration of Generative AI for Critical Thinking in Mobile Networks](https://mlsanity.com/paper/2404.06946/) (2024)
- [Technical Language Processing for Telecommunications Specifications](https://mlsanity.com/paper/2406.02325/) (2024)
- [Knowledge Distillation of Domain-adapted LLMs for Question-Answering in Telecom](https://mlsanity.com/paper/2504.20000/) (2025)
- [NEFMind: Parameter-Efficient Fine-Tuning of Open-Source LLMs for Telecom APIs Automation](https://mlsanity.com/paper/2508.09240/) (2025)
- [MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications](https://mlsanity.com/paper/2511.13131/) (2025)
- [CREST: Improving Interpretability and Effectiveness of Troubleshooting at Ericsson through Criterion-Specific Trouble Report Retrieval](https://mlsanity.com/paper/2511.17417/) (2025)
- [Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline](https://mlsanity.com/paper/2606.29014/) (2026)

## Abstract (arXiv)

The Third Generation Partnership Project (3GPP) has successfully introduced standards for global mobility. However, the volume and complexity of these standards has increased over time, thus complicating access to relevant information for vendors and service providers. Use of Generative Artificial Intelligence (AI) and in particular Large Language Models (LLMs), may provide faster access to relevant information. In this paper, we evaluate the capability of state-of-art LLMs to be used as Question Answering (QA) assistants for 3GPP document reference. Our contribution is threefold. First, we provide a benchmark and measuring methods for evaluating performance of LLMs. Second, we do data preprocessing and fine-tuning for one of these LLMs and provide guidelines to increase accuracy of the responses that apply to all LLMs. Third, we provide a model of our own, TeleRoBERTa, that performs on-par with foundation LLMs but with an order of magnitude less number of parameters. Results show that LLMs can be used as a credible reference tool on telecom technical documents, and thus have potential for a number of different applications from troubleshooting and maintenance, to network operations and software product development.