# STMI: Segmentation-Guided Token Modulation with Cross-Modal Hypergraph Interaction for Multi-Modal Object Re-Identification

arXiv: https://arxiv.org/abs/2603.00695 · HTML: https://mlsanity.com/paper/2603.00695/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association](https://mlsanity.com/paper/2103.02440/) (2021)
- [VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing](https://mlsanity.com/paper/2502.17258/) (2025)
- [Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.04337/) (2021)
- [HAT: Hierarchical Aggregation Transformers for Person Re-identification](https://mlsanity.com/paper/2107.05946/) (2021)
- [Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2403.10254/) (2024)
- [TOP-ReID: Multi-spectral Object Re-Identification with Token Permutation](https://mlsanity.com/paper/2312.09612/) (2023)
- [DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2412.10650/) (2024)
- [IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2503.10324/) (2025)
- [MambaPro: Multi-Modal Object Re-Identification with Mamba Aggregation and Synergistic Prompt](https://mlsanity.com/paper/2412.10707/) (2024)
- [UniCat: Crafting a Stronger Fusion Baseline for Multimodal Re-Identification](https://mlsanity.com/paper/2310.18812/) (2023)
- [Multi-spectral Vehicle Re-identification with Cross-directional Consistency Network and a High-quality Benchmark](https://mlsanity.com/paper/2208.00632/) (2022)
- [MaskReID: A Mask Based Deep Ranking Neural Network for Person Re-identification](https://mlsanity.com/paper/1804.03864/) (2018)
- [ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identification](https://mlsanity.com/paper/2505.17821/) (2025)
- [NEXT: Multi-Grained Mixture of Experts via Text-Modulation for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2505.20001/) (2025)
- [DMPT: Decoupled Modality-aware Prompt Tuning for Multi-modal Object Re-identification](https://mlsanity.com/paper/2504.10985/) (2025)
- [Reliable Multi-Modal Object Re-Identification via Modality-Aware Graph Reasoning](https://mlsanity.com/paper/2504.14847/) (2025)
- [UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2507.04638/) (2025)

## Built on by (most cited first, 3 of 3)

- [Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation](https://mlsanity.com/paper/2607.29207/) (2026)
- [Incentive Noise and Structural Prior Infusion for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2609.24539/) (2026)
- [TRIM-ReID: Duplication-Aware Token Reduction and Modality-Aligned Interaction for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2610.04361/) (2026)

## Abstract (arXiv)

Multi-modal object Re-Identification (ReID) aims to exploit complementary information from different modalities to retrieve specific objects. However, existing methods often rely on hard token filtering or simple fusion strategies, which can lead to the loss of discriminative cues and increased background interference. To address these challenges, we propose STMI, a novel multi-modal learning framework consisting of three key components: (1) Segmentation-Guided Feature Modulation (SFM) module leverages SAM-generated masks to enhance foreground representations and suppress background noise through learnable attention modulation; (2) Semantic Token Reallocation (STR) module employs learnable query tokens and an adaptive reallocation mechanism to extract compact and informative representations without discarding any tokens; (3) Cross-Modal Hypergraph Interaction (CHI) module constructs a unified hypergraph across modalities to capture high-order semantic relationships. Extensive experiments on public benchmarks (i.e., RGBNT201, RGBNT100, and MSVR310) demonstrate the effectiveness and robustness of our proposed STMI framework in multi-modal ReID scenarios.