mlsanityData to 14 Oct 2026

Safety Signals to Verify NetOps Agents with Action-Level Granularity

arXiv 2609.14422 ↗First posted 13 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Agentic Network Operations (NetOps) are an emerging paradigm promising to enable workload-aware, self-adjustable, and reliable autonomous networks. While agents have proven their value in incident summarization and telemetry signal extraction, their effectiveness as autonomous control-loop engines heavily relies on their long-horizon reliability. One such setting is the datacenter fabric, where an agent must respond to alarms and operator intents while abstaining from high-risk actions that may cause or extend downtime. Abstention, however, presupposes that an action's impact is known pre-execution, which necessitates a per-action ground truth that NetOps agent benchmarks do not provide. We construct such a ground truth for the network repair task of NetArena. A symbolic replay of the emulated network, validated against the environment at every turn, yields the exact value of every action. From the action-level value, we derive two pre-execution targets, namely whether an action reduces the repair distance (progress) and whether it increases it (harm). We show across 10 agent models, that agent verifiers leveraging internal signals predict both harm and progress more reliably than a baseline using observable signals only. Perspectively, we aim to use these signals as safety feedback to an agent harness to abstain from risky actions and protect the target system.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 49 of its 69 references that are in the map. First posted on arXiv 13 Sep 2026.

Builds on

49 of 69 references.

References · 49

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  12. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
37 more
  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  6. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  7. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  11. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  12. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  13. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  14. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  15. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  16. 2026Gemma 4 Technical Reportcited by 259
  17. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  18. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  19. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  20. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  21. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  22. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  23. 2019Hindsight Credit Assignmentcited by 79
  24. 2020Aligning Superhuman AI with Human Behavior: Chess as a Model Systemcited by 79
  25. 2024NetLLM: Adapting Large Language Models for Networkingcited by 78
  26. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 77
  27. 2025NVIDIA Nemotron 3: Efficient and Open Intelligencecited by 65
  28. 2024AutoPSV: Automated Process-Supervised Verifiercited by 30
  29. 2023Large Language Models for Networking: Applications, Enabling Techniques, and Challengescited by 28
  30. 2025Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMscited by 15
  31. 2025NetArena: Dynamic Benchmarks for AI Agents in Network Automationcited by 9
  32. 2025A Network Arena for Benchmarking AI Agents on Network Troubleshootingcited by 9
  33. 2026TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modelingcited by 8
  34. 2026From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agentscited by 3
  35. 2026AgentLens: Interpretable Safety Steering via Mechanistic Subspaces for Multi-Turn Coding Agentcited by 3
  36. 2026Predicting Future Behaviors in Reasoning Models Enables Better Steeringcited by 2
  37. 2026PreAct-Bench: Benchmarking Predictive Monitoring in LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Agentic Executions Fail: Detecting and Localizing Runtime Faults from Telemetry
  2. 2026NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration
  3. 2026Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions
  4. 2026Autonomous Incident Resolution at Hyperscale: An Agentic AI Architecture for Network Operations
  5. 2026Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy
  6. 2026Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety
  7. 2026LogAct: Enabling Agentic Reliability via Shared Logs
  8. 2025NetMoniAI: An Agentic AI Framework for Network Security & Monitoring