TeleTables: A Benchmark for Large Language Models in Telecom Table Interpretation
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are increasingly applied to telecom engineering tasks, yet perform poorly on 3GPP specifications. These standards encode much of their technical information in complex tables, but LLM knowledge and interpretation of such tables remain largely unexplored. We introduce TeleTables, a benchmark comprising 2,220 tables from 13 3GPP specifications in four formats and 500 human-verified MCQs spanning direct retrieval to multi-step reasoning. Evaluating 20 open-weight LLMs across non reasoning, multimodal, reasoning, and table specialized architectures reveals two distinct performance bottlenecks. In the closed-book setting, domain knowledge is the primary constraint, with no general-purpose model exceeding 41% accuracy. When the table is provided as context, the best models exceed 90%, but performance degrades systematically with reasoning depth, evidence scope, and structural complexity, with a 32.2pp spread across reasoning skills. Table specialization on non-telecom data provides no consistent benefit, while strong reasoning capabilities remain essential for reliable interpretation of complex technical tables.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 7 of its 14 references that are in the map. First posted on arXiv 5 Dec 2025.
Builds on
7 of 14 references.
References · 7
- 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
- 2023Large Language Models for Telecom: Forthcoming Impact on the Industrycited by 44
- 2024TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specificationscited by 26
- 2025Table-R1: Inference-Time Scaling for Table Reasoningcited by 23
- 2025TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answeringcited by 14
- 2025Chat3GPP: An Open-Source Retrieval-Augmented Generation Framework for 3GPP Documentscited by 5
- 2025Telco-oRAG: Optimizing Retrieval-augmented Generation for Telecom Queries via Hybrid Retrieval and Neural Routingcited by 3
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 20266G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning with Foundation Models in AI-Native 6G Networkscited by 6
- 2026TeleResilienceBench: Quantifying Resilience for LLM Reasoning in Telecommunicationscited by 2
- 2026TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?cited by 1
- 2026TelecomGPT-R1: A Unified Open-Source Reasoner for the Telecom Stackcited by 0
- 2026TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Taskscited by 0
- 2026TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks21 Sep 2026
- 2026TelecomGPT-R1: A Unified Open-Source Reasoner for the Telecom Stack22 Jun 2026
- 2026TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?18 May 2026
- 2026TeleResilienceBench: Quantifying Resilience for LLM Reasoning in Telecommunications11 May 2026
- 20266G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning with Foundation Models in AI-Native 6G Networks9 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2024Telecom Language Models: Must They Be Large?
- 2026Enhancing Large Language Models (LLMs) for Telecom using Dynamic Knowledge Graphs and Explainable Retrieval-Augmented Generation
- 2025TeleMath: A Benchmark for Large Language Models in Telecom Mathematical Problem Solving
- 2025MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications
- 2025TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models
- 2025SafeCOMM: A Study on Safety Degradation in Fine-Tuned Telecom Large Language Models
- 2025Evaluating Open-Source Large Language Models for Technical Telecom Question Answering
- 2025RUST-BENCH: Benchmarking LLM Reasoning on Unstructured Text within Structured Tables