# DCMM-Transformer: Degree-Corrected Mixed-Membership Attention for Medical Imaging

arXiv: https://arxiv.org/abs/2511.12047 · HTML: https://mlsanity.com/paper/2511.12047/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 15 Nov 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Mixed Membership Estimation for Social Networks](https://mlsanity.com/paper/1708.07852/) (2017)
- [Evaluation of Community Detection Methods](https://mlsanity.com/paper/1807.01130/) (2018)
- [Ocular Recognition Databases and Competitions: A Survey](https://mlsanity.com/paper/1911.09646/) (2019)
- [Core-Periphery Principle Guided Redesign of Self-Attention in Transformers](https://mlsanity.com/paper/2303.15569/) (2023)
- [Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers](https://mlsanity.com/paper/2204.09225/) (2022)
- [Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost](https://mlsanity.com/paper/2210.15541/) (2022)
- [Semi-Supervised Multimodal Multi-Instance Learning for Aortic Stenosis Diagnosis](https://mlsanity.com/paper/2403.06024/) (2024)
- [Community Detection in Complex Networks using Link Prediction](https://mlsanity.com/paper/1611.00254/) (2016)
- [Domain-Adaptive Diagnosis of Lewy Body Disease with Transferability Aware Transformer](https://mlsanity.com/paper/2507.08839/) (2025)

## Abstract (arXiv)

Medical images exhibit latent anatomical groupings, such as organs, tissues, and pathological regions, that standard Vision Transformers (ViTs) fail to exploit. While recent work like SBM-Transformer attempts to incorporate such structures through stochastic binary masking, they suffer from non-differentiability, training instability, and the inability to model complex community structure. We present DCMM-Transformer, a novel ViT architecture for medical image analysis that incorporates a Degree-Corrected Mixed-Membership (DCMM) model as an additive bias in self-attention. Unlike prior approaches that rely on multiplicative masking and binary sampling, our method introduces community structure and degree heterogeneity in a fully differentiable and interpretable manner. Comprehensive experiments across diverse medical imaging datasets, including brain, chest, breast, and ocular modalities, demonstrate the superior performance and generalizability of the proposed approach. Furthermore, the learned group structure and structured attention modulation substantially enhance interpretability by yielding attention maps that are anatomically meaningful and semantically coherent.