# Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification

arXiv: https://arxiv.org/abs/2403.10254 · HTML: https://mlsanity.com/paper/2403.10254/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 60 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [Learning Discriminative Features with Multiple Granularities for Person Re-Identification](https://mlsanity.com/paper/1804.01438/) (2018)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [Adaptive Token Sampling For Efficient Vision Transformers](https://mlsanity.com/paper/2111.15667/) (2021)
- [TransFG: A Transformer Architecture for Fine-grained Recognition](https://mlsanity.com/paper/2103.07976/) (2021)
- [Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification](https://mlsanity.com/paper/2108.08728/) (2021)
- [TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval](https://mlsanity.com/paper/2207.07852/) (2022)
- [Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification](https://mlsanity.com/paper/2205.02151/) (2022)
- [Token Pooling in Vision Transformers](https://mlsanity.com/paper/2110.03860/) (2021)
- [Which Tokens to Use? Investigating Token Reduction in Vision Transformers](https://mlsanity.com/paper/2308.04657/) (2023)
- [Multi-Level Factorisation Net for Person Re-Identification](https://mlsanity.com/paper/1803.09132/) (2018)
- [NFormer: Robust Person Re-identification with Neighbor Transformer](https://mlsanity.com/paper/2204.09331/) (2022)
- [AAformer: Auto-Aligned Transformer for Person Re-Identification](https://mlsanity.com/paper/2104.00921/) (2021)
- [Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.04337/) (2021)
- [Multi-scale Deep Learning Architectures for Person Re-identification](https://mlsanity.com/paper/1709.05165/) (2017)

## Built on by (most cited first, 30 of 35)

- [DeMo: Decoupled Feature-Based Mixture of Experts for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2412.10650/) (2024)
- [IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2503.10324/) (2025)
- [MambaPro: Multi-Modal Object Re-Identification with Mamba Aggregation and Synergistic Prompt](https://mlsanity.com/paper/2412.10707/) (2024)
- [SeCap: Self-Calibrating and Adaptive Prompts for Cross-view Person Re-Identification in Aerial-Ground Networks](https://mlsanity.com/paper/2503.06965/) (2025)
- [Other Tokens Matter: Exploring Global and Local Features of Vision Transformers for Object Re-Identification](https://mlsanity.com/paper/2404.14985/) (2024)
- [Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2506.11036/) (2025)
- [Unity is Strength: Unifying Convolutional and Transformeral Features for Better Person Re-Identification](https://mlsanity.com/paper/2412.17239/) (2024)
- [LATex: Leveraging Attribute-based Text Knowledge for Aerial-Ground Person Re-Identification](https://mlsanity.com/paper/2503.23722/) (2025)
- [ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identification](https://mlsanity.com/paper/2505.17821/) (2025)
- [Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification](https://mlsanity.com/paper/2501.16811/) (2025)
- [SD-ReID: View-aware Stable Diffusion for Aerial-Ground Person Re-Identification](https://mlsanity.com/paper/2504.09549/) (2025)
- [MDReID: Modality-Decoupled Learning for Any-to-Any Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2510.23301/) (2025)
- [NEXT: Multi-Grained Mixture of Experts via Text-Modulation for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2505.20001/) (2025)
- [GSAlign: Geometric and Semantic Alignment Network for Aerial-Ground Person Re-Identification](https://mlsanity.com/paper/2510.22268/) (2025)
- [Signal: Selective Interaction and Global-local Alignment for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2511.17965/) (2025)
- [A Theory-Inspired Framework for Few-Shot Cross-Modal Sketch Person Re-Identification](https://mlsanity.com/paper/2511.18677/) (2025)
- [DMPT: Decoupled Modality-aware Prompt Tuning for Multi-modal Object Re-identification](https://mlsanity.com/paper/2504.10985/) (2025)
- [Reliable Multi-Modal Object Re-Identification via Modality-Aware Graph Reasoning](https://mlsanity.com/paper/2504.14847/) (2025)
- [UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2507.04638/) (2025)
- [STMI: Segmentation-Guided Token Modulation with Cross-Modal Hypergraph Interaction for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2603.00695/) (2026)
- [Unsupervised Visible-Infrared ReID via Pseudo-label Correction and Modality-level Alignment](https://mlsanity.com/paper/2404.06683/) (2024)
- [SCING:Towards More Efficient and Robust Person Re-Identification through Selective Cross-modal Prompt Tuning](https://mlsanity.com/paper/2507.00506/) (2025)
- [ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation](https://mlsanity.com/paper/2607.06565/) (2026)
- [Collaborative Enhancement Network for Low-quality Multi-spectral Vehicle Re-identification](https://mlsanity.com/paper/2504.14877/) (2025)
- [Modality Unified Attack for Omni-Modality Person Re-Identification](https://mlsanity.com/paper/2501.12761/) (2025)
- [DCG ReID: Disentangling Collaboration and Guidance Fusion Representations for Multi-modal Vehicle Re-Identification](https://mlsanity.com/paper/2601.02924/) (2026)
- [ALADIN:Attribute-Language Distillation Network for Person Re-Identification](https://mlsanity.com/paper/2603.21482/) (2026)
- [FUSE: Frequency-domain Unification and Spectral Energy Alignment for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2606.20044/) (2026)
- [HiHR: Hierarchical Hyperbolic Representation for Aerial-Ground Person Re-Identification](https://mlsanity.com/paper/2607.09186/) (2026)
- [Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance](https://mlsanity.com/paper/2607.23451/) (2026)

## Abstract (arXiv)

Single-modal object re-identification (ReID) faces great challenges in maintaining robustness within complex visual scenarios. In contrast, multi-modal object ReID utilizes complementary information from diverse modalities, showing great potentials for practical applications. However, previous methods may be easily affected by irrelevant backgrounds and usually ignore the modality gaps. To address above issues, we propose a novel learning framework named EDITOR to select diverse tokens from vision Transformers for multi-modal object ReID. We begin with a shared vision Transformer to extract tokenized features from different input modalities. Then, we introduce a Spatial-Frequency Token Selection (SFTS) module to adaptively select object-centric tokens with both spatial and frequency information. Afterwards, we employ a Hierarchical Masked Aggregation (HMA) module to facilitate feature interactions within and across modalities. Finally, to further reduce the effect of backgrounds, we propose a Background Consistency Constraint (BCC) and an Object-Centric Feature Refinement (OCFR). They are formulated as two new loss functions, which improve the feature discrimination with background suppression. As a result, our framework can generate more discriminative features for multi-modal object ReID. Extensive experiments on three multi-modal ReID benchmarks verify the effectiveness of our methods. The code is available at https://github.com/924973292/EDITOR.