# Walk the Talk: Is Your Log-based Software Reliability Maintenance System Really Reliable?

arXiv: https://arxiv.org/abs/2509.24352 · HTML: https://mlsanity.com/paper/2509.24352/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2025.

## Built on by (most cited first, 10 of 10)

- [MicroRemed: Benchmarking LLMs in Microservices Remediation](https://mlsanity.com/paper/2511.01166/) (2025)
- [Agentic Memory Enhanced Recursive Reasoning for Root Cause Localization in Microservices](https://mlsanity.com/paper/2601.02732/) (2026)
- [Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation](https://mlsanity.com/paper/2603.21522/) (2026)
- [From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery](https://mlsanity.com/paper/2605.15412/) (2026)
- [E2E-REME: Towards End-to-End Microservices Auto-Remediation via Experience-Simulation Reinforcement Fine-Tuning](https://mlsanity.com/paper/2604.11094/) (2026)
- [Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuning](https://mlsanity.com/paper/2605.04431/) (2026)
- [Towards In-Depth Root Cause Localization for Microservices with Multi-Agent Recursion-of-Thought](https://mlsanity.com/paper/2605.14866/) (2026)
- [Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation](https://mlsanity.com/paper/2605.10988/) (2026)
- [Holmes: Multimodal Agentic Diagnosis for Mixed-Language Mobile Crashes at Industrial Scale](https://mlsanity.com/paper/2606.21963/) (2026)
- [Playing social deduction games with reinforcement fine-tuned large language models](https://mlsanity.com/paper/2610.04261/) (2026)

## Abstract (arXiv)

Log-based software reliability maintenance systems are crucial for sustaining stable customer experience. However, existing deep learning-based methods represent a black box for service providers, making it impossible for providers to understand how these methods detect anomalies, thereby hindering trust and deployment in real production environments. To address this issue, this paper defines a trustworthiness metric, diagnostic faithfulness, for models to gain service providers' trust, based on surveys of SREs at a major cloud provider. We design two evaluation tasks: attention-based root cause localization and event perturbation. Empirical studies demonstrate that existing methods perform poorly in diagnostic faithfulness. Consequently, we propose FaithLog, a faithful log-based anomaly detection system, which achieves faithfulness through a carefully designed causality-guided attention mechanism and adversarial consistency learning. Evaluation results on two public datasets and one industrial dataset demonstrate that the proposed method achieves state-of-the-art performance in diagnostic faithfulness.