mlsanityData to 14 Oct 2026

TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network

arXiv 2411.02617 ↗First posted 4 Nov 2024cs.CL cs.LG cs.NIcited by 2

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

The telecommunications industry's rapid evolution demands intelligent systems capable of managing complex networks and adapting to emerging technologies. While large language models (LLMs) show promise in addressing these challenges, their deployment in telecom environments faces significant constraints due to edge device limitations and inconsistent documentation. To bridge this gap, we present TeleOracle, a telecom-specialized retrieval-augmented generation (RAG) system built on the Phi-2 small language model (SLM). To improve context retrieval, TeleOracle employs a two-stage retriever that incorporates semantic chunking and hybrid keyword and semantic search. Additionally, we expand the context window during inference to enhance the model's performance on open-ended queries. We also employ low-rank adaption for efficient fine-tuning. A thorough analysis of the model's performance indicates that our RAG framework is effective in aligning Phi-2 to the telecom domain in a downstream question and answer (QnA) task, achieving a 30% improvement in accuracy over the base Phi-2 model, reaching an overall accuracy of 81.20%. Notably, we show that our model not only performs on par with the much larger LLMs but also achieves a higher faithfulness score, indicating higher adherence to the retrieved context.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 38 of its 55 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

38 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024QMOS: Enhancing LLMs for Telecommunication with Question Masked loss and Option Shuffling21 Sep
  2. 2024Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards21 Aug
  3. 2024Evaluation of RAG Metrics for Question Answering in the Telecom Domain15 Jul
  4. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence16 Apr

Other references · 34

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  8. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  9. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  10. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  11. 2023Large Language Model Unlearningcited by 302
  12. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
22 more
  1. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  2. 2023TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledgecited by 63
  3. 2022An Analysis of Fusion Functions for Hybrid Retrievalcited by 51
  4. 2023Large Language Models for Telecom: Forthcoming Impact on the Industrycited by 44
  5. 2023CLEX: Continuous Length Extrapolation for Large Language Modelscited by 44
  6. 2024Interactive AI with Retrieval-Augmented Generation for Next Generation Networkingcited by 34
  7. 2023Understanding Telecom Language Through Large Language Modelscited by 32
  8. 2023NetGPT: Generative Pretrained Transformer for Network Trafficcited by 31
  9. 2024TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specificationscited by 27
  10. 2024Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunicationscited by 26
  11. 2024Tele-LLMs: A Series of Specialized Large Language Models for Telecommunicationscited by 25
  12. 2024ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networkscited by 24
  13. 2023SPEC5G: A Dataset for 5G Cellular Network Protocol Analysiscited by 20
  14. 2023Injecting the BM25 Score as Text Improves BERT-Based Re-rankerscited by 17
  15. 2024Using Large Language Models to Understand Telecom Standardscited by 13
  16. 2024TelecomRAG: Taming Telecom Standards with Retrieval Augmented Generation and LLMscited by 13
  17. 2023Observations on LLMs for Telecom Domain: Capabilities and Limitationscited by 11
  18. 2024Linguistic Intelligence in Large Language Models for Telecommunicationscited by 11
  19. 2024Telecom Language Models: Must They Be Large?cited by 10
  20. 2024Large Language Model-Driven Curriculum Design for Mobile Networkscited by 8
  21. 2024A Primer on Generative AI for Telecom: From Theory to Practicecited by 3
  22. 2024Large Language Model-aided Edge Learning in Distribution System State Estimationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAGcited by 16
  2. 2026An Agentic AI Framework with Large Language Models and Chain-of-Thought for UAV-Assisted Logistics Scheduling with Mobile Edge Computingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026ARMOR: Adaptive Retriever Optimization for Low-Resource Telecom Question Answering
  2. 2025Telco-oRAG: Optimizing Retrieval-augmented Generation for Telecom Queries via Hybrid Retrieval and Neural Routing
  3. 2025Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation
  4. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  5. 2025How to Discover Knowledge for FutureG: Contextual RAG and LLM Prompting for O-RAN
  6. 2025Retrieval-augmented reasoning with lean language models
  7. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation
  8. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning