mlsanityData to 7 Oct 2026

Human Grounded Evaluation of Large Language Models for Optical Network Automation

arXiv 2607.18068 ↗First posted 20 Jul 2026cs.NI cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly adopted for network automation, yet their output quality and inference cost can vary substantially across LLM families. We present HuGLEN, a stepwise evaluation pipeline that uses an LLM-as-a-judge together with a small set of expert ratings to enable scalable and reproducible comparison of candidate LLMs, and to rank them using a quality efficiency score (QES). We demonstrate HuGLEN for translating outputs from an explainable artificial intelligence (XAI) model for the optical network quality of transmission (QoT) estimation task into operator-friendly explanations. Our results show that a medium-sized LLM (12B parameters) achieves the highest QES, indicating the best trade-off between explanation quality and efficiency. Overall, HuGLEN reduces the human-labeling burden while supporting consistent model selection for operator-facing automation tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 25 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

15 of 25 references.

References · 15

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  6. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  7. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
  8. 2024Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Taskscited by 102
  9. 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 73
  10. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 54
  11. 2024Tele-LLMs: A Series of Specialized Large Language Models for Telecommunicationscited by 25
  12. 2024First Field Trial of LLM-Powered AI Agent for Lifecycle Management of Autonomous Driving Optical Networkscited by 8
3 more
  1. 2024Synergistic Interplay of Large Language Model and Digital Twin for Autonomous Optical Networks: Field Demonstrationscited by 5
  2. 2025DeepForm: Reasoning Large Language Model for Communication System Formulationcited by 3
  3. 2026Generative Explainability for Next-Generation Networks: LLM-Augmented XAI with Mutual Feature Interactionscited by 1

Built on by

Not cited yet.