mlsanityData to 7 Oct 2026

Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles

arXiv 2306.00989 ↗First posted 1 Jun 2023cs.CV cs.LGcited by 287

Not on the map. Why?

Abstract (arXiv)

Modern hierarchical vision transformers have added several vision-specific components in the pursuit of supervised classification performance. While these components lead to effective accuracies and attractive FLOP counts, the added complexity actually makes these transformers slower than their vanilla ViT counterparts. In this paper, we argue that this additional bulk is unnecessary. By pretraining with a strong visual pretext task (MAE), we can strip out all the bells-and-whistles from a state-of-the-art multi-stage vision transformer without losing accuracy. In the process, we create Hiera, an extremely simple hierarchical vision transformer that is more accurate than previous models while being significantly faster both at inference and during training. We evaluate Hiera on a variety of tasks for image and video recognition. Our code and models are available at https://github.com/facebookresearch/hiera.

Key facts. Cited by 287 papers in the map (data through 7 Oct 2026). Builds on 50 of its 66 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

50 of 66 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2017Mask R-CNNcited by 7,259
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  11. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  3. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  4. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2018SlowFast Networks for Video Recognitioncited by 1,862
  7. 2021Vision Transformers for Dense Predictioncited by 1,789
  8. 2021ViViT: A Video Vision Transformercited by 1,610
  9. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  10. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  12. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  13. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  14. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  15. 2021Scaling Vision Transformerscited by 993
  16. 2016Deep Networks with Stochastic Depthcited by 973
  17. 2021Multiscale Vision Transformerscited by 896
  18. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  19. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  20. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  21. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  22. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  23. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  24. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  25. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  26. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  27. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  28. 2022DeiT III: Revenge of the ViTcited by 417
  29. 2018A Short Note about Kinetics-600cited by 331
  30. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  31. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  32. 2022Masked Autoencoders that Listencited by 271
  33. 2021Are Large-scale Datasets Necessary for Self-Supervised Pre-training?cited by 117
  34. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 107
  35. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  36. 2022MILAN: Masked Image Pretraining on Language Assisted Representationcited by 69
  37. 2022Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Localitycited by 57
  38. 2022Green Hierarchical Vision Transformer for Masked Image Modelingcited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 72; 2025: 136; 2026: 69.

287 papers.

Sort:

By area:

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  2. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  3. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  4. 2025MedSAM2: Segment Anything in 3D Medical Images and Videoscited by 88
  5. 2024Streaming Dense Video Captioningcited by 82
  6. 2024When Do We Not Need Larger Vision Models?cited by 65
  7. 2024SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentationcited by 63
  8. 2024A Distractor-Aware Memory for Visual Object Tracking with SAM2cited by 63
  9. 2024Memory Consolidation Enables Long-Context Video Understandingcited by 45
  10. 2024DINO-Foresight: Looking into the Future with DINOcited by 44
  11. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 43
  12. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 42
38 more
  1. 2024Segment Anything in Medical Images and Videos: Benchmark and Deploymentcited by 42
  2. 2024Don't Look Twice: Faster Video Transformers with Run-Length Tokenizationcited by 42
  3. 2024Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objectscited by 35
  4. 2024CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentationcited by 33
  5. 2024Prithvi WxC: Foundation Model for Weather and Climatecited by 28
  6. 2024A Simple Baseline for Efficient Hand Mesh Reconstructioncited by 27
  7. 2025OmniSAM: Omnidirectional Segment Anything Model for UDA in Panoramic Semantic Segmentationcited by 26
  8. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  9. 2023Frequency-Aware Masked Autoencoders for Multimodal Pretraining on Biosignalscited by 22
  10. 2024MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detectioncited by 21
  11. 2025An Empirical Study of Autoregressive Pre-training from Videoscited by 21
  12. 2025EdgeTAM: On-Device Track Anything Modelcited by 21
  13. 2024VideoMAC: Video Masked Autoencoders Meet ConvNetscited by 20
  14. 2024Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Modelscited by 20
  15. 2024Efficient Track Anythingcited by 20
  16. 2023CHAMMI: A benchmark for channel-adaptive models in microscopy imagingcited by 18
  17. 2023Window Attention is Bugged: How not to Interpolate Position Embeddingscited by 18
  18. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
  19. 2024Referring Atomic Video Action Recognitioncited by 16
  20. 2024LookupViT: Compressing visual information to a limited number of tokenscited by 16
  21. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
  22. 2024ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentationcited by 16
  23. 2024ZIM: Zero-Shot Image Matting for Anythingcited by 15
  24. 2024Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Expertscited by 15
  25. 2025SignRep: Enhancing Self-Supervised Sign Representationscited by 15
  26. 2025ReSurgSAM2: Referring Segment Anything in Surgical Video via Credible Long-term Trackingcited by 15
  27. 2024VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videoscited by 14
  28. 2025OpenWorldSAM: Extending SAM2 for Universal Image Segmentation with Language Promptscited by 14
  29. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  30. 2025Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fieldscited by 13
  31. 2025H3DP: Triply-Hierarchical Diffusion Policy for Visuomotor Learningcited by 13
  32. 2025Towards Scalable Language-Image Pre-training for 3D Medical Imagingcited by 13
  33. 2024VideoEval: Comprehensive Benchmark Suite for Low-Cost Evaluation of Video Foundation Modelcited by 12
  34. 2025CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for Multi-Organ Segmentationcited by 12
  35. 2025Object-centric Video Question Answering with Visual Grounding and Referringcited by 12
  36. 2024GenRec: Unifying Video Generation and Recognition with Diffusion Modelscited by 11
  37. 2025Towards Visuospatial Cognition via Hierarchical Fusion of Visual Expertscited by 11
  38. 2024Computer Vision for Primate Behavior Analysis in the Wildcited by 10

50 most cited of 287 All 287 (JSON)