mlsanityData to 14 Oct 2026

Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model

arXiv 2105.15089 ↗First posted 31 May 2021cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Inspired by biological evolution, we explain the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) and derive that both of them have consistent mathematical representation. Analogous to the dynamic local population in EA, we improve the existing transformer structure and propose a more efficient EAT model, and design task-related heads to deal with different tasks more flexibly. Moreover, we introduce the spatial-filling curve into the current vision transformer to sequence image data into a uniform sequential format. Thus we can design a unified EAT framework to address multi-modal tasks, separating the network architecture from the data format adaptation. Our approach achieves state-of-the-art results on the ImageNet classification task compared with recent vision transformer works while having smaller parameters and greater throughput. We further conduct multi-modal tasks to demonstrate the superiority of the unified EAT, e.g., Text-Based Image Retrieval, and our approach improves the rank-1 by +3.7 points over the baseline on the CSS dataset.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 54 of its 85 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

54 of 85 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  11. 2018Deep contextualized word representationscited by 2,290
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
38 more
  1. 2020Reformer: The Efficient Transformercited by 1,668
  2. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  3. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  4. 2020Rethinking Attention with Performerscited by 1,601
  5. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  6. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  7. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  8. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  10. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  11. 2020Pre-Trained Image Processing Transformercited by 893
  12. 2021Going deeper with Image Transformerscited by 764
  13. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
  14. 2021Transformer in Transformercited by 633
  15. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  16. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  17. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 419
  18. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  19. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  20. 2017Population Based Training of Neural Networkscited by 367
  21. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  22. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 334
  23. 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 320
  24. 2021DeepViT: Towards Deeper Vision Transformercited by 289
  25. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  26. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  27. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  28. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  29. 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
  30. 2017Automatic Spatially-aware Fashion Concept Discoverycited by 134
  31. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  32. 2021SiT: Self-supervised vIsion Transformercited by 99
  33. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 83
  34. 2019CARS: Continuous Evolution for Efficient Neural Architecture Searchcited by 79
  35. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  36. 2020RealFormer: Transformer Likes Residual Attentioncited by 62
  37. 2021Can Vision Transformers Learn without Natural Images?cited by 32
  38. 2021Facial Attribute Transformers for Precise and Robust Makeup Transfercited by 17

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 5; 2025: 0; 2026: 1.

20 papers.

Sort:

By area:

  1. 2024MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detectioncited by 113
  2. 2022Towards Robust Referring Image Segmentationcited by 49
  3. 2023Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detectioncited by 47
  4. 2022Region-Aware Face Swappingcited by 45
  5. 2023Rethinking Mobile Block for Efficient Attention-based Modelscited by 36
  6. 2023Large Language Model for Multi-objective Evolutionary Optimizationcited by 29
  7. 2023Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentationcited by 28
  8. 2022EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithmcited by 27
  9. 2024When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challengescited by 20
  10. 2023B2Opt: Learning to Optimize Black-box Optimization with Little Budgetcited by 15
  11. 2024Pretrained Optimization Model for Zero-Shot Black Box Optimizationcited by 13
  12. 2023Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentationcited by 12
8 more
  1. 2023JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recoverycited by 12
  2. 2023PVG: Progressive Vision Graph for Vision Recognitioncited by 10
  3. 2023Toward High Quality Facial Representation Learningcited by 10
  4. 2023Better "CMOS" Produces Clearer Images: Learning Space-Variant Blur Estimation for Blind Image Super-Resolutioncited by 7
  5. 2024A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detectioncited by 7
  6. 2023Learning Global-aware Kernel for Image Harmonizationcited by 4
  7. 2024Exploring Real&Synthetic Dataset and Linear Attention in Image Restorationcited by 3
  8. 2026Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AIcited by 0