# CONSULT: Contrastive Self-Supervised Learning for Few-shot Tumor Detection

arXiv: https://arxiv.org/abs/2410.11307 · HTML: https://mlsanity.com/paper/2410.11307/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows](https://mlsanity.com/paper/2111.07677/) (2021)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies](https://mlsanity.com/paper/2303.14535/) (2023)
- [Registration based Few-Shot Anomaly Detection](https://mlsanity.com/paper/2207.07361/) (2022)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection](https://mlsanity.com/paper/1901.08954/) (2019)
- [Inpainting Transformer for Anomaly Detection](https://mlsanity.com/paper/2104.13897/) (2021)
- [Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detection](https://mlsanity.com/paper/2207.01463/) (2022)
- [AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentation](https://mlsanity.com/paper/2110.10403/) (2021)
- [CarveMix: A Simple Data Augmentation Method for Brain Lesion Segmentation](https://mlsanity.com/paper/2108.06883/) (2021)
- [AnoViT: Unsupervised Anomaly Detection and Localization with Vision Transformer-based Encoder-Decoder](https://mlsanity.com/paper/2203.10808/) (2022)

## Abstract (arXiv)

Artificial intelligence aids in brain tumor detection via MRI scans, enhancing the accuracy and reducing the workload of medical professionals. However, in scenarios with extremely limited medical images, traditional deep learning approaches tend to fail due to the absence of anomalous images. Anomaly detection also suffers from ineffective feature extraction due to vague training process. Our work introduces a novel two-stage anomaly detection algorithm called CONSULT (CONtrastive Self-sUpervised Learning for few-shot Tumor detection). The first stage of CONSULT fine-tunes a pre-trained feature extractor specifically for MRI brain images, using a synthetic data generation pipeline to create tumor-like data. This process overcomes the lack of anomaly samples and enables the integration of attention mechanisms to focus on anomalous image segments. The first stage is to overcome the shortcomings of current anomaly detection in extracting features in high-variation data by incorporating Context-Aware Contrastive Learning and Self-supervised Feature Adversarial Learning. The second stage of CONSULT uses PatchCore for conventional feature extraction via the fine-tuned weights from the first stage. To summarize, we propose a self-supervised training scheme for anomaly detection, enhancing model performance and data reliability. Furthermore, our proposed contrastive loss, Tritanh Loss, stabilizes learning by offering a unique solution all while enhancing gradient flow. Finally, CONSULT achieves superior performance in few-shot brain tumor detection, demonstrating significant improvements over PatchCore by 9.4%, 12.9%, 10.2%, and 6.0% for 2, 4, 6, and 8 shots, respectively, while training exclusively on healthy images.