# Spectro-ViT: A Vision Transformer Model for GABA-edited MRS Reconstruction Using Spectrograms

arXiv: https://arxiv.org/abs/2311.15386 · HTML: https://mlsanity.com/paper/2311.15386/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 71 references that are in the map.
- First posted on arXiv 26 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Medical Image Segmentation Using Squeeze-and-Expansion Transformers](https://mlsanity.com/paper/2105.09511/) (2021)
- [ESResNet: Environmental Sound Classification Based on Visual Domain Models](https://mlsanity.com/paper/2004.07301/) (2020)

## Abstract (arXiv)

Purpose: To investigate the use of a Vision Transformer (ViT) to reconstruct/denoise GABA-edited magnetic resonance spectroscopy (MRS) from a quarter of the typically acquired number of transients using spectrograms. Theory and Methods: A quarter of the typically acquired number of transients collected in GABA-edited MRS scans are pre-processed and converted to a spectrogram image representation using the Short-Time Fourier Transform (STFT). The image representation of the data allows the adaptation of a pre-trained ViT for reconstructing GABA-edited MRS spectra (Spectro-ViT). The Spectro-ViT is fine-tuned and then tested using textitin vivo GABA-edited MRS data. The Spectro-ViT performance is compared against other models in the literature using spectral quality metrics and estimated metabolite concentration values. Results: The Spectro-ViT model significantly outperformed all other models in four out of five quantitative metrics (mean squared error, shape score, GABA+/water fit error, and full width at half maximum). The metabolite concentrations estimated (GABA+/water, GABA+/Cr, and Glx/water) were consistent with the metabolite concentrations estimated using typical GABA-edited MRS scans reconstructed with the full amount of typically collected transients. Conclusion: The proposed Spectro-ViT model achieved state-of-the-art results in reconstructing GABA-edited MRS, and the results indicate these scans could be up to four times faster.