# REPORT-OPS: Hybrid Evidence-Guided Reporting with Human-Supervised Refinement for HPC

arXiv: https://arxiv.org/abs/2610.19222 · HTML: https://mlsanity.com/paper/2610.19222/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Automatic Root Cause Analysis via Large Language Models for Cloud Incidents](https://mlsanity.com/paper/2305.15778/) (2023)
- [TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use](https://mlsanity.com/paper/2510.04550/) (2025)
- [From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents](https://mlsanity.com/paper/2606.04990/) (2026)
- [Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents](https://mlsanity.com/paper/2510.02837/) (2025)
- [VLA-Forget: Vision-Language-Action Unlearning for Embodied Foundation Models](https://mlsanity.com/paper/2604.03956/) (2026)
- [CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMs](https://mlsanity.com/paper/2603.21524/) (2026)
- [RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models](https://mlsanity.com/paper/2603.14819/) (2026)
- [G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMs](https://mlsanity.com/paper/2604.00419/) (2026)
- [AgentDevel: Reframing Self-Evolving LLM Agents as Release Engineering](https://mlsanity.com/paper/2601.04620/) (2026)
- [SafeCommit: Certifying When Memory-Grounded Agents May Safely Act](https://mlsanity.com/paper/2608.04289/) (2026)
- [ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents](https://mlsanity.com/paper/2606.18037/) (2026)
- [KORAL: Knowledge Graph Guided LLM Reasoning for SSD Operational Analysis](https://mlsanity.com/paper/2602.10246/) (2026)
- [Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs](https://mlsanity.com/paper/2609.10439/) (2026)

## Abstract (arXiv)

Operational reporting for high-performance computing (HPC) facilities requires combining information from monitoring services, schedulers, incident and maintenance records, storage measurements, periodic reports, and operator investigations. Producing a reliable report requires more than retrieving relevant records: the evidence must refer to the correct system, metric definition, time period, and source of authority. We present REPORT-OPS, a hybrid reporting agent in which an LLM interprets requests, plans evidence collection, evaluates possible explanations, and drafts reports, while author-seeded operational policies and automated checks determine whether the available evidence supports the requested conclusion. Depending on the evidence state, the system answers the request, reports a qualified conclusion, or withholds an unsupported claim. During offline refinement, an LLM critic proposes bounded policy changes that a human approves or rejects without editing; accepted candidates are promoted only after automated target, held-out, and regression testing. We evaluate REPORT-OPS on 68 sanitized reporting queries constructed from 50 base scenarios in a frozen HPC operational corpus, using three agent models and two LLM judges. REPORT-OPS achieves 68.63% Pass@80, compared with 7.02% for the strongest retrieval baseline. Evidence gating increases claim-support precision from 40.58% to 90.06%. On the 13-query explanatory subset, explanation accuracy reaches 55.54%. Across five rounds of human-supervised refinement, localized policy updates correct 91.17% of targeted failures and improve performance on related held-out cases by 9.33 percentage points, with no observed regression breaks under release gating. These results demonstrate improved evidence-grounded reporting and within-environment policy reuse; they do not establish unrestricted causal discovery or cross-facility generalization.