# Hierarchical Characterization of Brain Dynamics via State Space-based Vector Quantization

arXiv: https://arxiv.org/abs/2506.22952 · HTML: https://mlsanity.com/paper/2506.22952/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 28 Jun 2025.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation](https://mlsanity.com/paper/2205.12465/) (2022)
- [Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis](https://mlsanity.com/paper/2003.10613/) (2020)
- [Online Clustered Codebook](https://mlsanity.com/paper/2307.15139/) (2023)
- [BrainMass: Advancing Brain Network Analysis for Diagnosis with Large-scale Self-Supervised Learning](https://mlsanity.com/paper/2403.01433/) (2024)
- [BolT: Fused Window Transformers for fMRI Time Series Analysis](https://mlsanity.com/paper/2205.11578/) (2022)
- [Representation Collapsing Problems in Vector Quantization](https://mlsanity.com/paper/2411.16550/) (2024)
- [Advancing Brain Imaging Analysis Step-by-step via Progressive Self-paced Learning](https://mlsanity.com/paper/2407.16128/) (2024)
- [BPCLIP: A Bottom-up Image Quality Assessment from Distortion to Semantics Based on CLIP](https://mlsanity.com/paper/2506.17969/) (2025)
- [LVPNet: A Latent-variable-based Prediction-driven End-to-end Framework for Lossless Compression of Medical Images](https://mlsanity.com/paper/2506.17983/) (2025)

## Built on by (most cited first, 2 of 2)

- [BrainVista: Modeling Naturalistic Brain Dynamics as Multimodal Next-Token Prediction](https://mlsanity.com/paper/2602.04512/) (2026)
- [Progressive Self-Supervised Learning with Individualized Community Assignment for Brain Network Analysis](https://mlsanity.com/paper/2606.29695/) (2026)

## Abstract (arXiv)

Understanding brain dynamics through functional Magnetic Resonance Imaging (fMRI) remains a fundamental challenge in neuroscience, particularly in capturing how the brain transitions between various functional states. Recently, metastability, which refers to temporarily stable brain states, has offered a promising paradigm to quantify complex brain signals into interpretable, discretized representations. In particular, compared to cluster-based machine learning approaches, tokenization approaches leveraging vector quantization have shown promise in representation learning with powerful reconstruction and predictive capabilities. However, most existing methods ignore brain transition dependencies and lack a quantification of brain dynamics into representative and stable embeddings. In this study, we propose a Hierarchical State space-based Tokenization network, termed HST, which quantizes brain states and transitions in a hierarchical structure based on a state space-based model. We introduce a refined clustered Vector-Quantization Variational AutoEncoder (VQ-VAE) that incorporates quantization error feedback and clustering to improve quantization performance while facilitating metastability with representative and stable token representations. We validate our HST on two public fMRI datasets, demonstrating its effectiveness in quantifying the hierarchical dynamics of the brain and its potential in disease diagnosis and reconstruction performance. Our method offers a promising framework for the characterization of brain dynamics, facilitating the analysis of metastability.