# Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications

arXiv: https://arxiv.org/abs/2409.05314 · HTML: https://mlsanity.com/paper/2409.05314/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Builds on (most cited first, 30 of 33)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [Continual Pre-Training of Large Language Models: How to (re)warm your model?](https://mlsanity.com/paper/2308.04014/) (2023)
- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [Investigating Continual Pretraining in Large Language Models: Insights and Implications](https://mlsanity.com/paper/2402.17400/) (2024)
- [Large Language Models for Telecom: Forthcoming Impact on the Industry](https://mlsanity.com/paper/2308.06013/) (2023)
- [TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models](https://mlsanity.com/paper/2407.09424/) (2024)
- [Efficient Continual Pre-training for Building Domain Specific Large Language Models](https://mlsanity.com/paper/2311.08545/) (2023)
- [Understanding Telecom Language Through Large Language Models](https://mlsanity.com/paper/2306.07933/) (2023)
- [Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systems](https://mlsanity.com/paper/2402.01748/) (2024)
- [The Power of Large Language Models for Wireless Communication System Development: A Case Study on FPGA Platforms](https://mlsanity.com/paper/2307.07319/) (2023)
- [SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similarity](https://mlsanity.com/paper/2401.17072/) (2024)

## Built on by (most cited first, 25 of 25)

- [A Survey on Large Language Models for Communication, Network, and Service Management: Application Insights, Challenges, and Future Directions](https://mlsanity.com/paper/2412.19823/) (2024)
- [SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging](https://mlsanity.com/paper/2503.17239/) (2025)
- [ORANSight-2.0: Foundational LLMs for O-RAN](https://mlsanity.com/paper/2503.05200/) (2025)
- [Chat3GPP: An Open-Source Retrieval-Augmented Generation Framework for 3GPP Documents](https://mlsanity.com/paper/2501.13954/) (2025)
- [WirelessMathBench: A Mathematical Modeling Benchmark for LLMs in Wireless Communications](https://mlsanity.com/paper/2505.14354/) (2025)
- [Telco-oRAG: Optimizing Retrieval-augmented Generation for Telecom Queries via Hybrid Retrieval and Neural Routing](https://mlsanity.com/paper/2505.11856/) (2025)
- [WirelessMathLM: Teaching Mathematical Reasoning for LLMs in Wireless Communications with Reinforcement Learning](https://mlsanity.com/paper/2509.23219/) (2025)
- [TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network](https://mlsanity.com/paper/2411.02617/) (2024)
- [TN-AutoRCA: Benchmark Construction and Agentic Framework for Self-Improving Alarm-Based Root Cause Analysis in Telecommunication Networks](https://mlsanity.com/paper/2507.18190/) (2025)
- [QoEReasoner: An Agentic Reasoning Framework for Automated and Explainable QoE Diagnosis in RANs](https://mlsanity.com/paper/2606.01925/) (2026)
- [Distilling Large Language Models for Network Active Queue Management](https://mlsanity.com/paper/2501.16734/) (2025)
- [TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks](https://mlsanity.com/paper/2609.25356/) (2026)
- [Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Framework](https://mlsanity.com/paper/2501.09631/) (2025)
- [SafeCOMM: A Study on Safety Degradation in Fine-Tuned Telecom Large Language Models](https://mlsanity.com/paper/2506.00062/) (2025)
- [DeepSpecs: Expert-Level Questions Answering in 5G](https://mlsanity.com/paper/2511.01305/) (2025)
- [MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications](https://mlsanity.com/paper/2511.13131/) (2025)
- [TelcoAI: Advancing 3GPP Technical Specification Search through Agentic Multi-Modal Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.16984/) (2025)
- [Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights](https://mlsanity.com/paper/2605.10808/) (2026)
- [RadioMaster: Multi-Agent System for Autonomous Radio Signal Generation](https://mlsanity.com/paper/2606.01862/) (2026)
- [ARMOR: Adaptive Retriever Optimization for Low-Resource Telecom Question Answering](https://mlsanity.com/paper/2606.29706/) (2026)
- [Human Grounded Evaluation of Large Language Models for Optical Network Automation](https://mlsanity.com/paper/2607.18068/) (2026)
- [CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations](https://mlsanity.com/paper/2608.12002/) (2026)
- [SignalReasoner: Assessing the Upper Bound of 3B Models for Signal Mathematical Reasoning](https://mlsanity.com/paper/2608.17301/) (2026)
- [Free-Text Evaluation of LLMs for 5G Domain Knowledge and Fault Analysis using LLM-as-Judge](https://mlsanity.com/paper/2608.21021/) (2026)
- [TelecomGPT-R1: A Unified Open-Source Reasoner for the Telecom Stack](https://mlsanity.com/paper/2608.26126/) (2026)

## Abstract (arXiv)

The emergence of large language models (LLMs) has significantly impacted various fields, from natural language processing to sectors like medicine and finance. However, despite their rapid proliferation, the applications of LLMs in telecommunications remain limited, often relying on general-purpose models that lack domain-specific specialization. This lack of specialization results in underperformance, particularly when dealing with telecommunications-specific technical terminology and their associated mathematical representations. This paper addresses this gap by first creating and disseminating Tele-Data, a comprehensive dataset of telecommunications material curated from relevant sources, and Tele-Eval, a large-scale question-and-answer dataset tailored to the domain. Through extensive experiments, we explore the most effective training techniques for adapting LLMs to the telecommunications domain, ranging from examining the division of expertise across various telecommunications aspects to employing parameter-efficient techniques. We also investigate how models of different sizes behave during adaptation and analyze the impact of their training data on this behavior. Leveraging these findings, we develop and open-source Tele-LLMs, the first series of language models ranging from 1B to 8B parameters, specifically tailored for telecommunications. Our evaluations demonstrate that these models outperform their general-purpose counterparts on Tele-Eval and telecommunications-related literature tasks while retaining their previously acquired capabilities, thus avoiding the catastrophic forgetting phenomenon.