# Transformer-based end-to-end classification of variable-length volumetric data

arXiv: https://arxiv.org/abs/2307.06666 · HTML: https://mlsanity.com/paper/2307.06666/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 13 Jul 2023.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [FlexiViT: One Model for All Patch Sizes](https://mlsanity.com/paper/2212.08013/) (2022)
- [3D Deep Learning on Medical Images: A Review](https://mlsanity.com/paper/2004.00218/) (2020)
- [Deep learning-based survival prediction for multiple cancer types using histopathology images](https://mlsanity.com/paper/1912.07354/) (2019)
- [A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation](https://mlsanity.com/paper/2111.13300/) (2021)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics](https://mlsanity.com/paper/2209.11195/) (2022)
- [Context-Aware Transformers For Spinal Cancer Detection and Radiological Grading](https://mlsanity.com/paper/2206.13173/) (2022)
- [Video Representation Learning Using Discriminative Pooling](https://mlsanity.com/paper/1803.10628/) (2018)

## Built on by (most cited first, 2 of 2)

- [Predicting Outcomes in Video Games with Long Short Term Memory Networks](https://mlsanity.com/paper/2402.15923/) (2024)
- [High-Resolution Magnetic Particle Imaging System Matrix Recovery Using a Vision Transformer with Residual Feature Network](https://mlsanity.com/paper/2511.02212/) (2025)

## Abstract (arXiv)

The automatic classification of 3D medical data is memory-intensive. Also, variations in the number of slices between samples is common. Na"ive solutions such as subsampling can solve these problems, but at the cost of potentially eliminating relevant diagnosis information. Transformers have shown promising performance for sequential data analysis. However, their application for long sequences is data, computationally, and memory demanding. In this paper, we propose an end-to-end Transformer-based framework that allows to classify volumetric data of variable length in an efficient fashion. Particularly, by randomizing the input volume-wise resolution(#slices) during training, we enhance the capacity of the learnable positional embedding assigned to each volume slice. Consequently, the accumulated positional information in each positional embedding can be generalized to the neighbouring slices, even for high-resolution volumes at the test time. By doing so, the model will be more robust to variable volume length and amenable to different computational budgets. We evaluated the proposed approach in retinal OCT volume classification and achieved 21.96% average improvement in balanced accuracy on a 9-class diagnostic task, compared to state-of-the-art video transformers. Our findings show that varying the volume-wise resolution of the input during training results in more informative volume representation as compared to training with fixed number of slices per volume.