mlsanityData to 13 Oct 2026

An Empirical Study of NetOps Capability of Pre-Trained Large Language Models

arXiv 2309.05557 ↗First posted 11 Sep 2023cs.CL cs.AI cs.NIcited by 9

Not on the map. Why?

Abstract (arXiv)

Nowadays, the versatile capabilities of Pre-trained Large Language Models (LLMs) have attracted much attention from the industry. However, some vertical domains are more interested in the in-domain capabilities of LLMs. For the Networks domain, we present NetEval, an evaluation set for measuring the comprehensive capabilities of LLMs in Network Operations (NetOps). NetEval is designed for evaluating the commonsense knowledge and inference ability in NetOps in a multi-lingual context. NetEval consists of 5,732 questions about NetOps, covering five different sub-domains of NetOps. With NetEval, we systematically evaluate the NetOps capability of 26 publicly available LLMs. The results show that only GPT-4 can achieve a performance competitive to humans. However, some open models like LLaMA 2 demonstrate significant potential.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 11 of its 19 references that are in the map. First posted on arXiv 11 Sep 2023.

Builds on

11 of 19 references.

References · 11

  1. 2023GPT-4 Technical Reportcited by 18,278
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  8. 2017Billion-scale similarity search with GPUscited by 2,490
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,029
  10. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 896
  11. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 760

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 2; 2026: 1.

9 papers.

  1. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
  2. 2024WirelessLLM: Empowering Large Language Models Towards Wireless Intelligencecited by 43
  3. 2024SECURE: Benchmarking Large Language Models for Cybersecuritycited by 14
  4. 2024Linguistic Intelligence in Large Language Models for Telecommunicationscited by 11
  5. 2024GeNet: A Multimodal LLM-Based Co-Pilot for Network Topology and Configurationcited by 6
  6. 2026EntWorld: A Holistic Environment and Benchmark for Verifiable Enterprise GUI Agentscited by 6
  7. 2023OpsEval: A Comprehensive IT Operations Benchmark Suite for Large Language Modelscited by 5
  8. 2025AthenaBench: A Dynamic Benchmark for Evaluating LLMs in Cyber Threat Intelligencecited by 4
  9. 2025InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editingcited by 2