# Learning Collaborative Knowledge with Multimodal Representation for Polyp Re-Identification

arXiv: https://arxiv.org/abs/2408.05914 · HTML: https://mlsanity.com/paper/2408.05914/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 12 Aug 2024.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Learning Discriminative Features with Multiple Granularities for Person Re-Identification](https://mlsanity.com/paper/1804.01438/) (2018)
- [Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro](https://mlsanity.com/paper/1701.07717/) (2017)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [Eigen-CAM: Class Activation Map using Principal Components](https://mlsanity.com/paper/2008.00299/) (2020)
- [Identity-Guided Human Semantic Parsing for Person Re-Identification](https://mlsanity.com/paper/2007.13467/) (2020)
- [Weakly-Supervised Video Moment Retrieval via Semantic Completion Network](https://mlsanity.com/paper/1911.08199/) (2019)
- [Mixed High-Order Attention Network for Person Re-Identification](https://mlsanity.com/paper/1908.05819/) (2019)
- [NFormer: Robust Person Re-identification with Neighbor Transformer](https://mlsanity.com/paper/2204.09331/) (2022)
- [ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning](https://mlsanity.com/paper/1908.07410/) (2019)
- [Surpassing Real-World Source Training Data: Random 3D Characters for Generalizable Person Re-Identification](https://mlsanity.com/paper/2006.12774/) (2020)
- [VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/2008.10238/) (2020)
- [Incomplete Descriptor Mining with Elastic Loss for Person Re-Identification](https://mlsanity.com/paper/2008.04010/) (2020)
- [Spatio-temporal Video Re-localization by Warp LSTM](https://mlsanity.com/paper/1905.03922/) (2019)
- [Rethinking Person Re-Identification via Semantic-Based Pretraining](https://mlsanity.com/paper/2110.05074/) (2021)
- [Colo-SCRL: Self-Supervised Contrastive Representation Learning for Colonoscopic Video Retrieval](https://mlsanity.com/paper/2303.15671/) (2023)
- [Colo-ReID: Discriminative Representation Embedding with Meta-learning for Colonoscopic Polyp Re-Identification](https://mlsanity.com/paper/2308.00929/) (2023)

## Built on by (most cited first, 1 of 1)

- [ST-ColoNet: Spatio-Temporal Colon Segment Recognition via Hybrid Attention and Edge-Guided Feature Learning](https://mlsanity.com/paper/2605.28119/) (2026)

## Abstract (arXiv)

Colonoscopic Polyp Re-Identification aims to match the same polyp from a large gallery with images from different views taken using different cameras, which plays an important role in the prevention and treatment of colorectal cancer in computer-aided diagnosis. However, traditional methods for object ReID directly adopting CNN models trained on the ImageNet dataset usually produce unsatisfactory retrieval performance on colonoscopic datasets due to the large domain gap. Worsely, these solutions typically learn unimodal modal representations on the basis of visual samples, which fails to explore complementary information from other different modalities. To address this challenge, we propose a novel Deep Multimodal Collaborative Learning framework named DMCL for polyp re-identification, which can effectively encourage multimodal knowledge collaboration and reinforce generalization capability in medical scenarios. On the basis of it, a dynamic multimodal feature fusion strategy is introduced to leverage the optimized visual-text representations for multimodal fusion via end-to-end training. Experiments on the standard benchmarks show the benefits of the multimodal setting over state-of-the-art unimodal ReID models, especially when combined with the collaborative multimodal fusion strategy. The code is publicly available at https://github.com/JeremyXSC/DMCL.