mlsanityData to 7 Oct 2026

TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models

arXiv 2407.09424 ↗First posted 12 Jul 2024eess.SP cs.AIcited by 38

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have the potential to revolutionize the Sixth Generation (6G) communication networks. However, current mainstream LLMs generally lack the specialized knowledge in telecom domain. In this paper, for the first time, we propose a pipeline to adapt any general purpose LLMs to a telecom-specific LLMs. We collect and build telecom-specific pre-train dataset, instruction dataset, preference dataset to perform continual pre-training, instruct tuning and alignment tuning respectively. Besides, due to the lack of widely accepted evaluation benchmarks in telecom domain, we extend existing evaluation benchmarks and proposed three new benchmarks, namely, Telecom Math Modeling, Telecom Open QnA and Telecom Code Tasks. These new benchmarks provide a holistic evaluation of the capabilities of LLMs including math modeling, Open-Ended question answering, code generation, infilling, summarization and analysis in telecom domain. Our fine-tuned LLM TelecomGPT outperforms state of the art (SOTA) LLMs including GPT-4, Llama-3 and Mistral in Telecom Math Modeling benchmark significantly and achieve comparable performance in various evaluation benchmarks such as TeleQnA, 3GPP technical documents classification, telecom code summary and generation and infilling.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 64 of its 76 references that are in the map. First posted on arXiv 12 Jul 2024.

Builds on

64 of 76 references.

References · 64

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  3. 2023Mistral 7Bcited by 3,065
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2024Mixtral of Expertscited by 1,785
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  7. 2023BloombergGPT: A Large Language Model for Financecited by 923
  8. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  10. 2022Galactica: A Large Language Model for Sciencecited by 736
  11. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  12. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  13. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  14. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  15. 2023The Falcon Series of Open Language Modelscited by 526
  16. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  17. 2023Llemma: An Open Language Model For Mathematicscited by 385
  18. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  19. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  20. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  21. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  22. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
  23. 2023Continual Pre-training of Language Modelscited by 194
  24. 2023Efficient Large Language Models: A Surveycited by 183
  25. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  26. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 147
  27. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  28. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 119
  29. 2024ChemLLM: A Chemical Large Language Modelcited by 100
  30. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92
  31. 2023Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunitiescited by 89
  32. 2023FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscited by 68
  33. 2023Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Modelscited by 67
  34. 2023FinGPT: Democratizing Internet-scale Data for Financial Large Language Modelscited by 65
  35. 2023TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledgecited by 62
  36. 2023Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Modelscited by 58
  37. 2019Zero-shot Text Classification With Generative Language Modelscited by 55
  38. 2023Large Generative AI Models for Telecom: The Next Big Thing?cited by 52

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 15; 2026: 18.

38 papers.

Sort:

By area:

  1. 2024Tele-LLMs: A Series of Specialized Large Language Models for Telecommunicationscited by 25
  2. 2024A Survey on Large Language Models for Communication, Network, and Service Management: Application Insights, Challenges, and Future Directionscited by 25
  3. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  4. 2024Diffusion Models as Network Optimizers: Explorations and Analysiscited by 13
  5. 2025ORANSight-2.0: Foundational LLMs for O-RANcited by 13
  6. 2026WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networkscited by 7
  7. 2026Agentic AI Empowered Intent-Based Networking for 6Gcited by 6
  8. 2025Chat3GPP: An Open-Source Retrieval-Augmented Generation Framework for 3GPP Documentscited by 5
  9. 2026RF-GPT: Teaching AI to See the Wireless Worldcited by 5
  10. 2025TeleMath: A Benchmark for Large Language Models in Telecom Mathematical Problem Solvingcited by 4
  11. 2024Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimizationcited by 3
  12. 2025Telco-oRAG: Optimizing Retrieval-augmented Generation for Telecom Queries via Hybrid Retrieval and Neural Routingcited by 3
26 more
  1. 2024NetOrchLLM: Mastering Wireless Network Orchestration with Large Language Modelscited by 2
  2. 2025Benchmarking Vector, Graph and Hybrid Retrieval Augmented Generation (RAG) Pipelines for Open Radio Access Networks (ORAN)cited by 2
  3. 2025MobiLLM: An Agentic AI Framework for Closed-Loop Threat Mitigation in 6G Open RANscited by 2
  4. 2025Large Model Enabled Embodied Intelligence for 6G Integrated Perception, Communication, and Computation Networkcited by 2
  5. 2025From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communicationscited by 1
  6. 2025Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmapcited by 1
  7. 2025Retrieval-Augmented Generation for Reliable Interpretation of Radio Regulationscited by 1
  8. 2025Think Less, Label Better: Multi-Stage Domain-Grounded Synthetic Data Generation for Fine-Tuning Large Language Models in Telecommunicationscited by 1
  9. 2026TeleDoCTR: Domain-Specific and Contextual Troubleshooting for Telecommunicationscited by 1
  10. 2026BeamAgent: LLM-Aided MIMO Beamforming with Decoupled Intent Parsing and Alternating Optimization for Joint Site Selection and Precodingcited by 1
  11. 2025Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Frameworkcited by 0
  12. 2025Knowledge Distillation of Domain-adapted LLMs for Question-Answering in Telecomcited by 0
  13. 2025MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applicationscited by 0
  14. 2026Customized User Plane Processing via Code Generating AI Agents for Next Generation Mobile Networkscited by 0
  15. 2026When Does Multimodal AI Help? Diagnostic Complementarity of Vision-Language Models and CNNs for Spectrum Management in Satellite-Terrestrial Networkscited by 0
  16. 2026Enhancing Confidence Estimation in Telco LLMs via Twin-Pass CoT-Ensemblingcited by 0
  17. 2026TeleEmbedBench: A Multi-Corpus Embedding Benchmark for RAG in Telecommunicationscited by 0
  18. 2026Agentic AI-Based Joint Computing and Networking via Mixture of Experts and Large Language Modelscited by 0
  19. 2026Enwar 3.0: An Agentic Multi-Modal LLM Orchestrator for Situation-Aware Beamforming, Blockage Prediction, and Handover Managementcited by 0
  20. 2026Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insightscited by 0
  21. 2026LLM-assisted gNB Parameter Configuration for Radio Access Networkcited by 0
  22. 2026SignalReasoner: Assessing the Upper Bound of 3B Models for Signal Mathematical Reasoningcited by 0
  23. 2026TelecomGPT-R1: A Unified Open-Source Reasoner for the Telecom Stackcited by 0
  24. 2026CRAFT: Fine-Tuning Pre-hoc Explainability in AI-native 6G RANcited by 0
  25. 2026Large Language Models (LLMs) for Telecom Root Cause Analysis (RCA): A Structured Reasoning Framework for Evidence-Grounded Diagnosiscited by 0
  26. 2026TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Taskscited by 0