TeleResilienceBench: Quantifying Resilience for LLM Reasoning in Telecommunications
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Deploying large language models in telecommunications requires more than task accuracy. In realistic workflows, a model may inherit partially completed reasoning from a prior step, an upstream agent, or its own earlier generation, and must continue that reasoning even when it is already going wrong. We introduce TeleResilienceBench, a benchmark that quantifies this capability, which we term reasoning resilience, across seven telecom sub-domains drawn from the GSMA Open-Telco LLM suite. Instances are constructed by collecting failures from a weak generator model, truncating the flawed reasoning trace at its midpoint, and asking a target model to continue and correct it. We propose the Correct Flip Rate (CFR) as a direct measure of successful recovery and evaluate eight models spanning the Qwen3.5, Gemma4, and Nemotron-3 families. Our results show that even the strongest model achieves a macro-average CFR of only 29.1%, and scale does not reliably improve resilience within families. Nemotron-3-nano 4b outperforms all Qwen3.5 variants including the 27b model and leads the auxiliary TeleMath numerical evaluation at 23.4% CR%, offering the best resilience-to-cost ratio in the set. A difficulty-stratified analysis further reveals that existing telecom benchmark difficulty labels reflect factual specificity rather than reasoning depth, suggesting that current evaluations measure knowledge coverage more than reasoning ability.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 20 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
18 of 20 references.
Closest ancestors · 3
Same direction, newest first
- 2025Lost at the Beginning of Reasoning27 Jun
- 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
- 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
Other references · 15
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
- 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 65
- 2023TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledgecited by 63
- 2024TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specificationscited by 27
- 2024ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networkscited by 24
- 2025Reasoning Language Models for Root Cause Analysis in 5G Wireless Networkscited by 13
- 2025Beyond Connectivity: An Open Architecture for AI-RAN Convergence in 6Gcited by 10
3 more
- 20266G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning with Foundation Models in AI-Native 6G Networkscited by 7
- 2025TeleTables: A Benchmark for Large Language Models in Telecom Table Interpretationcited by 6
- 2026TelcoAgent-Bench: A Multilingual Benchmark for Telecom AI Agentscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026TelecomGPT-R1: A Unified Open-Source Reasoner for the Telecom Stack
- 2026TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks
- 2026"I May Not Have Articulated Myself Clearly": Diagnosing Dynamic Instability in LLM Reasoning at Inference Time
- 2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoning
- 2026ReFlect: An Effective Harness System for Complex Long-Horizon LLM Reasoning
- 2025TeleMath: A Benchmark for Large Language Models in Telecom Mathematical Problem Solving
- 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
- 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs