# Understanding Telecom Language Through Large Language Models

arXiv: https://arxiv.org/abs/2306.07933 · HTML: https://mlsanity.com/paper/2306.07933/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [On the Road to 6G: Visions, Requirements, Key Technologies and Testbeds](https://mlsanity.com/paper/2302.14536/) (2023)
- [Zero-shot Text Classification With Generative Language Models](https://mlsanity.com/paper/1912.10165/) (2019)
- [Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition](https://mlsanity.com/paper/2010.03746/) (2020)
- [Multi-class Text Classification using BERT-based Active Learning](https://mlsanity.com/paper/2104.14289/) (2021)

## Built on by (most cited first, 30 of 32)

- [Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities](https://mlsanity.com/paper/2405.10825/) (2024)
- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [Large Language Models for Telecom: Forthcoming Impact on the Industry](https://mlsanity.com/paper/2308.06013/) (2023)
- [WirelessLLM: Empowering Large Language Models Towards Wireless Intelligence](https://mlsanity.com/paper/2405.17053/) (2024)
- [TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models](https://mlsanity.com/paper/2407.09424/) (2024)
- [TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications](https://mlsanity.com/paper/2406.01768/) (2024)
- [At the Dawn of Generative AI Era: A Tutorial-cum-Survey on New Frontiers in 6G Wireless Intelligence](https://mlsanity.com/paper/2402.18587/) (2024)
- [Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications](https://mlsanity.com/paper/2409.05314/) (2024)
- [AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems](https://mlsanity.com/paper/2311.05842/) (2023)
- [Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networks](https://mlsanity.com/paper/2309.13223/) (2023)
- [GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning](https://mlsanity.com/paper/2402.16631/) (2024)
- [Using Large Language Models to Understand Telecom Standards](https://mlsanity.com/paper/2404.02929/) (2024)
- [Linguistic Intelligence in Large Language Models for Telecommunications](https://mlsanity.com/paper/2402.15818/) (2024)
- [Can LLMs Understand Computer Networks? Towards a Virtual System Administrator](https://mlsanity.com/paper/2404.12689/) (2024)
- [Advanced Architectures Integrated with Agentic AI for Next-Generation Wireless Networks](https://mlsanity.com/paper/2502.01089/) (2025)
- [Telecom Language Models: Must They Be Large?](https://mlsanity.com/paper/2403.04666/) (2024)
- [Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards](https://mlsanity.com/paper/2408.11775/) (2024)
- [MoE2: Optimizing Collaborative Inference for Edge Large Language Models](https://mlsanity.com/paper/2501.09410/) (2025)
- [Chat3GPP: An Open-Source Retrieval-Augmented Generation Framework for 3GPP Documents](https://mlsanity.com/paper/2501.13954/) (2025)
- [LLM Agents as 6G Orchestrator: A Paradigm for Task-Oriented Physical-Layer Automation](https://mlsanity.com/paper/2410.03688/) (2024)
- [TelcoLM: collecting data, adapting, and benchmarking language models for the telecommunication domain](https://mlsanity.com/paper/2412.15891/) (2024)
- [ORAN-GUIDE: RAG-Driven Prompt Learning for LLM-Augmented Reinforcement Learning in O-RAN Network Slicing](https://mlsanity.com/paper/2506.00576/) (2025)
- [TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network](https://mlsanity.com/paper/2411.02617/) (2024)
- [Prompt-Tuned LLM-Augmented DRL for Dynamic O-RAN Network Slicing](https://mlsanity.com/paper/2506.00574/) (2025)
- [Evaluating Open-Source Large Language Models for Technical Telecom Question Answering](https://mlsanity.com/paper/2509.21949/) (2025)
- [TeleDoCTR: Domain-Specific and Contextual Troubleshooting for Telecommunications](https://mlsanity.com/paper/2601.00691/) (2026)
- [Convolutional vs Large Language Models for Software Log Classification in Edge-Deployable Cellular Network Testing](https://mlsanity.com/paper/2407.03759/) (2024)
- [Knowledge Distillation of Domain-adapted LLMs for Question-Answering in Telecom](https://mlsanity.com/paper/2504.20000/) (2025)
- [An LLM-Powered AI Agent Framework for Holistic IoT Traffic Interpretation](https://mlsanity.com/paper/2510.13925/) (2025)
- [MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications](https://mlsanity.com/paper/2511.13131/) (2025)

## Abstract (arXiv)

The recent progress of artificial intelligence (AI) opens up new frontiers in the possibility of automating many tasks involved in Telecom networks design, implementation, and deployment. This has been further pushed forward with the evolution of generative artificial intelligence (AI), including the emergence of large language models (LLMs), which is believed to be the cornerstone toward realizing self-governed, interactive AI agents. Motivated by this, in this paper, we aim to adapt the paradigm of LLMs to the Telecom domain. In particular, we fine-tune several LLMs including BERT, distilled BERT, RoBERTa and GPT-2, to the Telecom domain languages, and demonstrate a use case for identifying the 3rd Generation Partnership Project (3GPP) standard working groups. We consider training the selected models on 3GPP technical documents (Tdoc) pertinent to years 2009-2019 and predict the Tdoc categories in years 2020-2023. The results demonstrate that fine-tuning BERT and RoBERTa model achieves 84.6% accuracy, while GPT-2 model achieves 83% in identifying 3GPP working groups. The distilled BERT model with around 50% less parameters achieves similar performance as others. This corroborates that fine-tuning pretrained LLM can effectively identify the categories of Telecom language. The developed framework shows a stepping stone towards realizing intent-driven and self-evolving wireless networks from Telecom languages, and paves the way for the implementation of generative AI in the Telecom domain.