# Improved Anomaly Detection by Using the Attention-Based Isolation Forest

arXiv: https://arxiv.org/abs/2210.02558 · HTML: https://mlsanity.com/paper/2210.02558/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 66 references that are in the map.
- First posted on arXiv 5 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Multivariate Time-series Anomaly Detection via Graph Attention Network](https://mlsanity.com/paper/2009.02040/) (2020)
- [Deep Learning for Anomaly Detection: A Review](https://mlsanity.com/paper/2007.02500/) (2020)
- [An Attentive Survey of Attention Models](https://mlsanity.com/paper/1904.02874/) (2019)
- [Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection](https://mlsanity.com/paper/2111.09099/) (2021)
- [Dive into Deep Learning](https://mlsanity.com/paper/2106.11342/) (2021)
- [Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond](https://mlsanity.com/paper/2004.11154/) (2020)
- [Deep Isolation Forest for Anomaly Detection](https://mlsanity.com/paper/2206.06602/) (2022)
- [Extended Isolation Forest](https://mlsanity.com/paper/1811.02141/) (2018)
- [Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art](https://mlsanity.com/paper/2004.00433/) (2020)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Self-Supervised Masking for Unsupervised Anomaly Detection and Localization](https://mlsanity.com/paper/2205.06568/) (2022)
- [Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection](https://mlsanity.com/paper/2209.12148/) (2022)
- [Functional Isolation Forest](https://mlsanity.com/paper/1904.04573/) (2019)
- [Attention-based Random Forest and Contamination Model](https://mlsanity.com/paper/2201.02880/) (2022)
- [Neural Attention Models in Deep Learning: Survey and Taxonomy](https://mlsanity.com/paper/2112.05909/) (2021)
- [AGBoost: Attention-based Modification of Gradient Boosting Machine](https://mlsanity.com/paper/2207.05724/) (2022)
- [Attention and Self-Attention in Random Forests](https://mlsanity.com/paper/2207.04293/) (2022)
- [Understanding Attention: In Minds and Machines](https://mlsanity.com/paper/2012.02659/) (2020)

## Built on by (most cited first, 2 of 2)

- [Anomaly Detection Based on Isolation Mechanisms: A Survey](https://mlsanity.com/paper/2403.10802/) (2024)
- [SilIF: Silhouette-Augmented Isolation Forest for Unsupervised Transaction Fraud Detection](https://mlsanity.com/paper/2605.26135/) (2026)

## Abstract (arXiv)

A new modification of Isolation Forest called Attention-Based Isolation Forest (ABIForest) for solving the anomaly detection problem is proposed. It incorporates the attention mechanism in the form of the Nadaraya-Watson regression into the Isolation Forest for improving solution of the anomaly detection problem. The main idea underlying the modification is to assign attention weights to each path of trees with learnable parameters depending on instances and trees themselves. The Huber's contamination model is proposed to be used for defining the attention weights and their parameters. As a result, the attention weights are linearly depend on the learnable attention parameters which are trained by solving the standard linear or quadratic optimization problem. ABIForest can be viewed as the first modification of Isolation Forest, which incorporates the attention mechanism in a simple way without applying gradient-based algorithms. Numerical experiments with synthetic and real datasets illustrate outperforming results of ABIForest. The code of proposed algorithms is available.