# TeleMath: A Benchmark for Large Language Models in Telecom Mathematical Problem Solving

arXiv: https://arxiv.org/abs/2506.10674 · HTML: https://mlsanity.com/paper/2506.10674/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 16 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 14 of 14)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models](https://mlsanity.com/paper/2407.09424/) (2024)
- [What do LLMs need to Synthesize Correct Router Configurations?](https://mlsanity.com/paper/2307.04945/) (2023)
- [SPEC5G: A Dataset for 5G Cellular Network Protocol Analysis](https://mlsanity.com/paper/2301.09201/) (2023)
- [TrafficLLM: Enhancing Large Language Models for Network Traffic Analysis with Generic Traffic Representation](https://mlsanity.com/paper/2504.04222/) (2025)
- [Hermes: A Large Language Model Framework on the Journey to Autonomous Networks](https://mlsanity.com/paper/2411.06490/) (2024)
- [WirelessMathBench: A Mathematical Modeling Benchmark for LLMs in Wireless Communications](https://mlsanity.com/paper/2505.14354/) (2025)
- [LLM-Based Emulation of the Radio Resource Control Layer: Towards AI-Native RAN Protocols](https://mlsanity.com/paper/2505.16821/) (2025)

## Built on by (most cited first, 5 of 5)

- [TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks](https://mlsanity.com/paper/2609.25356/) (2026)
- [Data Trajectory Alignment for LLM Domain Adaptation: A Two-Phase Synthesis Framework for Telecommunications Mathematics](https://mlsanity.com/paper/2511.06776/) (2025)
- [SignalReasoner: Assessing the Upper Bound of 3B Models for Signal Mathematical Reasoning](https://mlsanity.com/paper/2608.17301/) (2026)
- [TelecomGPT-R1: A Unified Open-Source Reasoner for the Telecom Stack](https://mlsanity.com/paper/2608.26126/) (2026)
- [OTel: Open Telco AI Datasets, Benchmarks, and Models](https://mlsanity.com/paper/2610.07766/) (2026)

## Abstract (arXiv)

The increasing adoption of artificial intelligence in telecommunications has raised interest in the capability of Large Language Models (LLMs) to address domain-specific, mathematically intensive tasks. Although recent advancements have improved the performance of LLMs in general mathematical reasoning, their effectiveness within specialized domains, such as signal processing, network optimization, and performance analysis, remains largely unexplored. To address this gap, we introduce TeleMath, the first benchmark dataset specifically designed to evaluate LLM performance in solving mathematical problems with numerical solutions in the telecommunications domain. Comprising 500 question-answer (QnA) pairs, TeleMath covers a wide spectrum of topics in the telecommunications field. This paper outlines the proposed QnAs generation pipeline, starting from a selected seed of problems crafted by Subject Matter Experts. The evaluation of a wide range of open-source LLMs reveals that best performance on TeleMath is achieved by recent models explicitly designed for mathematical or logical reasoning. In contrast, general-purpose models, even those with a large number of parameters, often struggle with these challenges. We have released the dataset and the evaluation code to ease result reproducibility and support future research.