mlsanityData to 14 Oct 2026

Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding

arXiv 2311.15206 ↗First posted 26 Nov 2023cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

In precision agriculture, the detection and recognition of insects play an essential role in the ability of crops to grow healthy and produce a high-quality yield. The current machine vision model requires a large volume of data to achieve high performance. However, there are approximately 5.5 million different insect species in the world. None of the existing insect datasets can cover even a fraction of them due to varying geographic locations and acquisition costs. In this paper, we introduce a novel "Insect-1M" dataset, a game-changing resource poised to revolutionize insect-related foundation model training. Covering a vast spectrum of insect species, our dataset, including 1 million images with dense identification labels of taxonomy hierarchy and insect descriptions, offers a panoramic view of entomology, enabling foundation models to comprehend visual and semantic information about insects like never before. Then, to efficiently establish an Insect Foundation Model, we develop a micro-feature self-supervised learning method with a Patch-wise Relevant Attention mechanism capable of discerning the subtle differences among insect images. In addition, we introduce Description Consistency loss to improve micro-feature modeling via insect descriptions. Through our experiments, we illustrate the effectiveness of our proposed approach in insect modeling and achieve State-of-the-Art performance on standard benchmarks of insect-related tasks. Our Insect Foundation Model and Dataset promise to empower the next generation of insect-related vision models, bringing them closer to the ultimate goal of precision agriculture.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 51 of its 75 references that are in the map. First posted on arXiv 26 Nov 2023.

Builds on

51 of 75 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  10. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  11. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  12. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
38 more
  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2014Going Deeper with Convolutionscited by 4,241
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  11. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  13. 2018YOLOv3: An Incremental Improvementcited by 2,659
  14. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  15. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  16. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  17. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  18. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  19. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  20. 2021Scaling Vision Transformerscited by 999
  21. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  22. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  23. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  24. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  25. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 168
  26. 2017Single-Shot Refinement Neural Network for Object Detectioncited by 129
  27. 2023Equivariant Similarity for Vision-Language Foundation Modelscited by 52
  28. 2023Micron-BERT: BERT-based Facial Micro-Expression Recognitioncited by 43
  29. 2021BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in Semantic Scene Segmentationcited by 31
  30. 2022DirecFormer: A Directed Attention in Transformer Approach to Robust Action Recognitioncited by 31
  31. 2023FREDOM: Fairness Domain Adaptation Approach to Semantic Scene Understandingcited by 27
  32. 2022Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformercited by 21
  33. 2021Fine-Grained Zero-Shot Learning with DNA as Side Informationcited by 20
  34. 2023Fairness Continual Learning Approach to Semantic Scene Understanding in Open-World Environmentscited by 16
  35. 2022Two-Dimensional Quantum Material Identification via Self-Attention and Soft-labeling in Deep Learningcited by 15
  36. 2022OTAdapt: Optimal Transport-based Approach For Unsupervised Domain Adaptationcited by 12
  37. 2023Fairness in Visual Clustering: A Novel Transformer Clustering Approachcited by 10
  38. 2020Weakly Supervised Learning Guided by Activation Mapping Applied to a Novel Citrus Pest Benchmarkcited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 14; 2025: 11; 2026: 3.

29 papers.

Sort:

By area:

  1. 2024BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversitycited by 24
  2. 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settingscited by 20
  3. 2024CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scalecited by 18
  4. 2024QClusformer: A Quantum Transformer-based Framework for Unsupervised Visual Clusteringcited by 11
  5. 2024Quantum Visual Feature Encoding Revisitedcited by 10
  6. 2024Hierarchical Quantum Control Gates for Functional MRI Understandingcited by 8
  7. 2024FungiTastic: A multi-modal dataset and benchmark for image categorizationcited by 7
  8. 2024BRACTIVE: A Brain Activation Approach to Human Visual Brain Learningcited by 6
  9. 2024EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understandingcited by 6
  10. 2025Directed-Tokens: A Robust Multi-Modality Alignment Approach to Large Language-Vision Modelscited by 5
  11. 2024COBRA: A Continual Learning Approach to Vision-Brain Understandingcited by 4
  12. 2023Brainformer: Mimic Human Visual Brain Functions to Machine Vision Models via fMRIcited by 3
17 more
  1. 2024RAFA-Net: Region Attention Network For Food Items And Agricultural Stress Recognitioncited by 3
  2. 2024Towards Scalable Insect Monitoring: Ultra-Lightweight CNNs as On-Device Triggers for Insect Camera Trapscited by 3
  3. 2025Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoringcited by 3
  4. 2025MANGO: Multimodal Attention-based Normalizing Flow Approach to Fusion Learningcited by 3
  5. 2024Video-Based Autism Detection with Deep Learningcited by 2
  6. 2025BeetleVerse: A Study on Taxonomic Classification of Ground Beetlescited by 2
  7. 2026φ-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Modelscited by 2
  8. 2024Deep-Wide Learning Assistance for Insect Pest Classificationcited by 1
  9. 2024A Novel Dataset for Video-Based Neurodivergent Classification Leveraging Extra-Stimulatory Behaviorcited by 0
  10. 2025AquaMonitor: A multimodal multi-view image sequence dataset for real-life aquatic invertebrate biodiversity monitoringcited by 0
  11. 2025BioAnalyst: A Foundation Model for Biodiversitycited by 0
  12. 2025BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understandingcited by 0
  13. 2025Bridging Domain Gaps for Fine-Grained Moth Classification Through Expert-Informed Adaptation and Foundation Model Priorscited by 0
  14. 2025BioAutoML-NAS: An End-to-End AutoML Framework for Multimodal Insect Classification via Neural Architecture Search on Large-Scale Biodiversity Datacited by 0
  15. 2025Hard Samples, Bad Labels: Robust Loss Functions That Know When to Back Offcited by 0
  16. 2026PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interactioncited by 0
  17. 2026α-Graph: Attention-Infused Normalizing Flow Approach to Tractable Graph Modelingcited by 0