mlsanityData to 14 Oct 2026

Telco-DPR: A Hybrid Dataset for Evaluating Retrieval Models of 3GPP Technical Specifications

arXiv 2410.19790 ↗First posted 15 Oct 2024cs.IR cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper proposes a Question-Answering (QA) system for the telecom domain using 3rd Generation Partnership Project (3GPP) technical documents. Alongside, a hybrid dataset, Telco-DPR, which consists of a curated 3GPP corpus in a hybrid format, combining text and tables, is presented. Additionally, the dataset includes a set of synthetic question/answer pairs designed to evaluate the retrieval performance of QA systems on this type of data. The retrieval models, including the sparse model, Best Matching 25 (BM25), as well as dense models, such as Dense Passage Retriever (DPR) and Dense Hierarchical Retrieval (DHR), are evaluated and compared using top-K accuracy and Mean Reciprocal Rank (MRR). The results show that DHR, a retriever model utilising hierarchical passage selection through fine-tuning at both the document and passage levels, outperforms traditional methods in retrieving relevant technical information, achieving a Top-10 accuracy of 86.2%. Additionally, the Retriever-Augmented Generation (RAG) technique, used in the proposed QA system, is evaluated to demonstrate the benefits of using the hybrid dataset and the DHR. The proposed QA system, using the developed RAG model and the Generative Pretrained Transformer (GPT)-4, achieves a 14% improvement in answer accuracy, when compared to a previous benchmark on the same dataset.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

21 of 36 references.

References · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  3. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  4. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  5. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 229
  6. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  7. 2020Open Question Answering over Tables and Textcited by 163
  8. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  9. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  10. 2023TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledgecited by 63
  11. 2023Large Language Models for Telecom: Forthcoming Impact on the Industrycited by 44
  12. 2020A Survey on Machine Reading Comprehension Systemscited by 41
9 more
  1. 2021Dense Hierarchical Retrieval for Open-Domain Question Answeringcited by 34
  2. 2024TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specificationscited by 27
  3. 2024Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunicationscited by 26
  4. 2024Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Datacited by 24
  5. 2023SPEC5G: A Dataset for 5G Cellular Network Protocol Analysiscited by 20
  6. 2023AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systemscited by 19
  7. 2024TelecomRAG: Taming Telecom Standards with Retrieval Augmented Generation and LLMscited by 13
  8. 2024Telecom Language Models: Must They Be Large?cited by 10
  9. 2022Lecture Notes on Neural Information Retrievalcited by 7

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025A Graph-based Approach for Multi-Modal Question Answering from Flowcharts in Telecom Documentscited by 3
  2. 2025Retrieval-Augmented Generation for Reliable Interpretation of Radio Regulationscited by 1
  3. 2025TelcoAI: Advancing 3GPP Technical Specification Search through Agentic Multi-Modal Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluation of RAG Metrics for Question Answering in the Telecom Domain
  2. 2026ARMOR: Adaptive Retriever Optimization for Low-Resource Telecom Question Answering
  3. 2025LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generation
  4. 2024Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU
  5. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering
  6. 2024QMOS: Enhancing LLMs for Telecommunication with Question Masked loss and Option Shuffling
  7. 2025T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generation
  8. 2024Dense Passage Retrieval: Is it Retrieving?