mlsanityData to 14 Oct 2026

Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

arXiv 2307.06304 ↗First posted 12 Jul 2023cs.CV cs.AI cs.LGcited by 229

Not on the map. Why?

Cited by 63 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The ubiquitous and demonstrably suboptimal choice of resizing images to a fixed resolution before processing them with computer vision models has not yet been successfully challenged. However, models such as the Vision Transformer (ViT) offer flexible sequence-based modeling, and hence varying input sequence lengths. We take advantage of this with NaViT (Native Resolution ViT) which uses sequence packing during training to process inputs of arbitrary resolutions and aspect ratios. Alongside flexible model usage, we demonstrate improved training efficiency for large-scale supervised and contrastive image-text pretraining. NaViT can be efficiently transferred to standard tasks such as image and video classification, object detection, and semantic segmentation and leads to improved results on robustness and fairness benchmarks. At inference time, the input resolution flexibility can be used to smoothly navigate the test-time cost-performance trade-off. We believe that NaViT marks a departure from the standard, CNN-designed, input and modelling pipeline used by most computer vision models, and represents a promising direction for ViTs.

Key facts. Cited by 229 papers in the map (data through 14 Oct 2026). Builds on 48 of its 61 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

48 of 61 references.

References · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  10. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  12. 2017The Kinetics Human Action Video Datasetcited by 1,909
36 more
  1. 2021ViViT: A Video Vision Transformercited by 1,620
  2. 2019Natural Adversarial Examplescited by 1,408
  3. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  4. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2021Scaling Vision Transformerscited by 999
  7. 2021Multiscale Vision Transformerscited by 897
  8. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  9. 2021Robust fine-tuning of zero-shot modelscited by 797
  10. 2020Efficient Transformers: A Surveycited by 790
  11. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  12. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  13. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  14. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  15. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  16. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  17. 2019Measuring Calibration in Deep Learningcited by 446
  18. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  19. 2022DeiT III: Revenge of the ViTcited by 419
  20. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  21. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  22. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  23. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  24. 2019Fixing the train-test resolution discrepancycited by 209
  25. 2021Partial success in closing the gap between human and machine visioncited by 185
  26. 2021Self-attention Does Not Need O(n2) Memorycited by 154
  27. 2022FlexiViT: One Model for All Patch Sizescited by 152
  28. 2021Exploring the Limits of Large Scale Pre-trainingcited by 104
  29. 2021The Efficiency Misnomercited by 98
  30. 2019FairFace: Face Attribute Dataset for Balanced Race, Gender, and Agecited by 87
  31. 2021SCENIC: A JAX Library for Computer Vision Research and Beyondcited by 57
  32. 2019A Multigrid Method for Efficiently Training Video Modelscited by 52
  33. 2022Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learningcited by 49
  34. 2022Location-Aware Self-Supervised Transformers for Semantic Segmentationcited by 20
  35. 2022Auto-scaling Vision Transformers without Trainingcited by 15
  36. 2021A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 52; 2025: 92; 2026: 81.

229 papers.

Sort:

By area:

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  2. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  4. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  5. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  6. 2025Kimi-VL Technical Reportcited by 388
  7. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  8. 2024What matters when building vision-language models?cited by 326
  9. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  10. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  11. 2024Pixtral 12Bcited by 179
  12. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
38 more
  1. 2024Building and better understanding vision-language models: insights and future directionscited by 173
  2. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  3. 2025Seedream 3.0 Technical Reportcited by 154
  4. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 151
  5. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
  6. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
  7. 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 138
  8. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 132
  9. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 129
  10. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 116
  11. 2025Long Context Tuning for Video Generationcited by 113
  12. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 110
  13. 2024Multimodal Autoregressive Pre-training of Large Vision Encoderscited by 109
  14. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
  15. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 97
  16. 2024RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Modelscited by 89
  17. 2024FiT: Flexible Vision Transformer for Diffusion Modelcited by 85
  18. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  19. 2025Seaweed-7B: Cost-Effective Training of Video Generation Foundation Modelcited by 82
  20. 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 74
  21. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  22. 2025Kwai Keye-VL 1.5 Technical Reportcited by 73
  23. 2025Kling-Omni Technical Reportcited by 71
  24. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 68
  25. 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
  26. 2025Ovis-U1 Technical Reportcited by 67
  27. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  28. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 57
  29. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 53
  30. 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 49
  31. 2025Kwai Keye-VL Technical Reportcited by 47
  32. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 47
  33. 2024Don't Look Twice: Faster Video Transformers with Run-Length Tokenizationcited by 43
  34. 2024LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexitycited by 43
  35. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  36. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  37. 2025Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformerscited by 36
  38. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36

50 most cited of 229 All 229 (JSON)