# 6G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning with Foundation Models in AI-Native 6G Networks

arXiv: https://arxiv.org/abs/2602.08675 · HTML: https://mlsanity.com/paper/2602.08675/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios](https://mlsanity.com/paper/2511.11252/) (2025)
- [α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networks](https://mlsanity.com/paper/2601.03281/) (2026)
- [Toward 6G Native-AI Network: Foundation Model based Cloud-Edge-End Collaboration Framework](https://mlsanity.com/paper/2310.17471/) (2023)
- [Agentic AI Empowered Intent-Based Networking for 6G](https://mlsanity.com/paper/2601.06640/) (2026)
- [TeleTables: A Benchmark for Large Language Models in Telecom Table Interpretation](https://mlsanity.com/paper/2601.04202/) (2025)
- [Poisoning Attacks in Federated Edge Learning for Digital Twin 6G-enabled IoTs: An Anticipatory Study](https://mlsanity.com/paper/2303.11745/) (2023)
- [Large Generative Model-assisted Talking-face Semantic Communication System](https://mlsanity.com/paper/2411.03876/) (2024)
- [Towards Native Intelligence: 6G-LLM Trained with Reinforcement Learning from NDT Feedback](https://mlsanity.com/paper/2601.09992/) (2026)
- [RAG-Empowered LLM-Driven Dynamic Radio Resource Management in Open 6G RAN](https://mlsanity.com/paper/2511.22933/) (2025)
- [Advancing LLM-Based Security Automation with Customized Group Relative Policy Optimization for Zero-Touch Networks](https://mlsanity.com/paper/2512.09485/) (2025)
- [LLM-RIMSA: Large Language Models driven Reconfigurable Intelligent Metasurface Antenna Systems](https://mlsanity.com/paper/2508.12728/) (2025)
- [A Framework for AI-Native Semantic-Based Dynamic Slicing for 6G Networks](https://mlsanity.com/paper/2510.10756/) (2025)
- [CovertComBench: A First Domain-Specific Testbed for LLMs in Wireless Covert Communication](https://mlsanity.com/paper/2601.18315/) (2026)

## Built on by (most cited first, 7 of 7)

- [6GAgentGym: Tool Use, Data Synthesis, and Agentic Learning for Network Management](https://mlsanity.com/paper/2603.29656/) (2026)
- [How Small Can 6G Reason? Scaling Tiny-to-Small Language Models for AI-Native Networks](https://mlsanity.com/paper/2603.02156/) (2026)
- [6G Needs Agents: Toward Agentic AI-Native Networks for Autonomous Intelligence](https://mlsanity.com/paper/2605.01546/) (2026)
- [TeleResilienceBench: Quantifying Resilience for LLM Reasoning in Telecommunications](https://mlsanity.com/paper/2605.09929/) (2026)
- [AI-Native Closed-Loop Security for 6G-Enabled Cyber-Physical Systems: From Edge Detection to Network-Wide Mitigation](https://mlsanity.com/paper/2606.08173/) (2026)
- [Free-Text Evaluation of LLMs for 5G Domain Knowledge and Fault Analysis using LLM-as-Judge](https://mlsanity.com/paper/2608.21021/) (2026)
- [When LLM Agents See a Conflict but Still Proceed: Cross-Domain Dependencies in Network Slicing](https://mlsanity.com/paper/2610.17118/) (2026)

## Abstract (arXiv)

This paper introduces 6G-Bench, an open benchmark for evaluating semantic communication and network-level reasoning in AI-native 6G networks. 6G-Bench defines a taxonomy of 30 decision-making tasks (T1--T30) extracted from ongoing 6G and AI-agent standardization activities in 3GPP, IETF, ETSI, ITU-T, and the O-RAN Alliance, and organizes them into five standardization-aligned capability categories. Starting from 113,475 scenarios, we generate a balanced pool of 10,000 very-hard multiple-choice questions using task-conditioned prompts that enforce multi-step quantitative reasoning under uncertainty and worst-case regret minimization over multi-turn horizons. After automated filtering and expert human validation, 3,722 questions are retained as a high-confidence evaluation set, while the full pool is released to support training and fine-tuning of 6G-specialized models. Using 6G-Bench, we evaluate 22 foundation models spanning dense and mixture-of-experts architectures, short- and long-context designs (up to 1M tokens), and both open-weight and proprietary systems. Across models, deterministic single-shot accuracy (pass@1) spans a wide range from 0.22 to 0.82, highlighting substantial variation in semantic reasoning capability. Leading models achieve intent and policy reasoning accuracy in the range 0.87--0.89, while selective robustness analysis on reasoning-intensive tasks shows pass@5 values ranging from 0.20 to 0.91. To support open science and reproducibility, we release the 6G-Bench dataset on GitHub: https://github.com/maferrag/6G-Bench