mlsanityData to 7 Oct 2026

Attention Map Guided Transformer Pruning for Edge Device

arXiv 2304.01452 ↗First posted 4 Apr 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Due to its significant capability of modeling long-range dependencies, vision transformer (ViT) has achieved promising success in both holistic and occluded person re-identification (Re-ID) tasks. However, the inherent problems of transformers such as the huge computational cost and memory footprint are still two unsolved issues that will block the deployment of ViT based person Re-ID models on resource-limited edge devices. Our goal is to reduce both the inference complexity and model size without sacrificing the comparable accuracy on person Re-ID, especially for tasks with occlusion. To this end, we propose a novel attention map guided (AMG) transformer pruning method, which removes both redundant tokens and heads with the guidance of the attention map in a hardware-friendly way. We first calculate the entropy in the key dimension and sum it up for the whole map, and the corresponding head parameters of maps with high entropy will be removed for model size reduction. Then we combine the similarity and first-order gradients of key tokens along the query dimension for token importance estimation and remove redundant key and value tokens to further reduce the inference complexity. Comprehensive experiments on Occluded DukeMTMC and Market-1501 demonstrate the effectiveness of our proposals. For example, our proposed pruning strategy on ViT-Base enjoys textup29.4% textupFLOPs savings with textup0.2% drop on Rank-1 and textup0.4% improvement on mAP, respectively.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

32 of 50 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  6. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  8. 2019Importance Estimation for Neural Network Pruningcited by 531
  9. 2017Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)cited by 472
  10. 2021TransReID: Transformer-based Object Re-Identificationcited by 432
  11. 2019Bag of Tricks and A Strong Baseline for Deep Person Re-identificationcited by 429
  12. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
20 more
  1. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  2. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  3. 2021Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformercited by 175
  4. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  5. 2021Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformercited by 119
  6. 2020High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identificationcited by 105
  7. 2021Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentationcited by 95
  8. 2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
  9. 2022Unified Visual Transformer Compressioncited by 79
  10. 2018Occluded Person Re-identificationcited by 72
  11. 2020Pose-guided Visible Part Matching for Occluded Person ReIDcited by 69
  12. 2019Foreground-aware Pyramid Reconstruction for Alignment-free Occluded Person Re-identificationcited by 55
  13. 2021Learning Disentangled Representation Implicitly via Transformer for Occluded Person Re-Identificationcited by 44
  14. 2021Feature Erasing and Diffusion Network for Occluded Person Re-Identificationcited by 41
  15. 2022Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentationcited by 37
  16. 2019STNReID : Deep Convolutional Networks with Pairwise Spatial Transformer Networks for Partial Person Re-identificationcited by 23
  17. 2016Exploiting Web Images for Dataset Construction: A Domain Robust Approachcited by 22
  18. 2020Pedestrian Models for Autonomous Driving Part I: Low-Level Models, from Sensing to Trackingcited by 13
  19. 2021Searching for Efficient Multi-Stage Vision Transformerscited by 9
  20. 2021Neighbourhood-guided Feature Reconstruction for Occluded Person Re-Identificationcited by 7

Built on by

Not cited yet.