# A unified framework for detecting point and collective anomalies in operating system logs via collaborative transformers

arXiv: https://arxiv.org/abs/2512.23380 · HTML: https://mlsanity.com/paper/2512.23380/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 142 references that are in the map.
- First posted on arXiv 29 Dec 2025.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory](https://mlsanity.com/paper/2305.02437/) (2023)
- [Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics](https://mlsanity.com/paper/2008.06448/) (2020)
- [UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2211.11256/) (2022)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis](https://mlsanity.com/paper/2006.15955/) (2020)
- [Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning](https://mlsanity.com/paper/1802.00924/) (2018)
- [TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2009.02902/) (2020)
- [Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies](https://mlsanity.com/paper/2308.07610/) (2023)
- [Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection](https://mlsanity.com/paper/2107.05908/) (2021)
- [Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generation](https://mlsanity.com/paper/2303.17579/) (2023)
- [Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs](https://mlsanity.com/paper/2008.09340/) (2020)
- [Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation](https://mlsanity.com/paper/2102.10365/) (2021)
- [Deep Learning for Anomaly Detection in Log Data: A Survey](https://mlsanity.com/paper/2207.03820/) (2022)
- [Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection](https://mlsanity.com/paper/2008.13361/) (2020)
- [LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection](https://mlsanity.com/paper/2401.04749/) (2024)
- [Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data](https://mlsanity.com/paper/1411.5005/) (2014)
- [Multivariate Log-based Anomaly Detection for Distributed Database](https://mlsanity.com/paper/2406.07976/) (2024)
- [A2Log: Attentive Augmented Log Anomaly Detection](https://mlsanity.com/paper/2109.09537/) (2021)

## Abstract (arXiv)

Log anomaly detection is crucial for preserving the security of operating systems. Depending on the source of log data collection, various information is recorded in logs that can be considered log modalities. In light of this intuition, unimodal methods often struggle by ignoring the different modalities of log data. Meanwhile, multimodal methods fail to handle the interactions between these modalities. Applying multimodal sentiment analysis to log anomaly detection, we propose CoLog, a framework that collaboratively encodes logs utilizing various modalities. CoLog utilizes collaborative transformers and multi-head impressed attention to learn interactions among several modalities, ensuring comprehensive anomaly detection. To handle the heterogeneity caused by these interactions, CoLog incorporates a modality adaptation layer, which adapts the representations from different log modalities. This methodology enables CoLog to learn nuanced patterns and dependencies within the data, enhancing its anomaly detection capabilities. Extensive experiments demonstrate CoLog's superiority over existing state-of-the-art methods. Furthermore, in detecting both point and collective anomalies, CoLog achieves a mean precision of 99.63%, a mean recall of 99.59%, and a mean F1 score of 99.61% across seven benchmark datasets for log-based anomaly detection. The comprehensive detection capabilities of CoLog make it highly suitable for cybersecurity, system monitoring, and operational efficiency. CoLog represents a significant advancement in log anomaly detection, providing a sophisticated and effective solution to point and collective anomaly detection through a unified framework and a solution to the complex challenges automatic log data analysis poses. We also provide the implementation of CoLog at https://github.com/NasirzadehMoh/CoLog.