# Hierarchical Attention for Sparse Volumetric Anomaly Detection in Subclinical Keratoconus

arXiv: https://arxiv.org/abs/2512.03346 · HTML: https://mlsanity.com/paper/2512.03346/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers](https://mlsanity.com/paper/2106.10270/) (2021)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations](https://mlsanity.com/paper/2106.01548/) (2021)
- [Generative OpenMax for Multi-Class Open Set Classification](https://mlsanity.com/paper/1707.07418/) (2017)
- [Adversarial Reciprocal Points Learning for Open Set Recognition](https://mlsanity.com/paper/2103.00953/) (2021)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations](https://mlsanity.com/paper/1406.2639/) (2014)
- [GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints](https://mlsanity.com/paper/1807.06294/) (2018)
- [Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation](https://mlsanity.com/paper/2012.09279/) (2020)
- [Transformed CNNs: recasting pre-trained convolutional layers with self-attention](https://mlsanity.com/paper/2106.05795/) (2021)

## Built on by (most cited first, 1 of 1)

- [CFCH: Coarse-Fine Collaborative Hierarchical Learning for Anterior Segment Disease Analysis](https://mlsanity.com/paper/2609.32559/) (2026)

## Abstract (arXiv)

The detection of weak, spatially distributed anomalies in volumetric medical imaging remains challenging due to the difficulty of integrating subtle signals across non-adjacent regions. This study presents a controlled comparison of sixteen architectures spanning convolutional, hybrid, and transformer families for subclinical keratoconus detection from three-dimensional anterior segment optical coherence tomography (AS-OCT). The results demonstrate that hierarchical architectures achieve 21-23% higher sensitivity and specificity, particularly in the difficult subclinical regime, outperforming both convolutional neural networks (CNNs) and global-attention Vision Transformer (ViT) baselines. Mechanistic analyses indicate that this advantage arises from spatial scale alignment: hierarchical windowing produces effective receptive fields matched to the intermediate extent of subclinical abnormalities, avoiding the excessive locality observed in convolutional models and the diffuse integration characteristic of pure global attention. Attention-distance measurements show that subclinical cases require longer spatial integration than healthy or overtly pathological volumes, with hierarchical models exhibiting lower variance and more anatomically coherent focus. Representational similarity further indicates that hierarchical attention learns a distinct feature space that balances local structure sensitivity with flexible long-range interactions. Auxiliary age and sex prediction tasks demonstrate moderately high cross-task consistency, supporting the generalizability of these inductive principles. The findings provide design guidance for volumetric anomaly detection and highlight hierarchical attention as a principled approach for early pathological change analysis in medical imaging.