mlsanityData to 7 Oct 2026

Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images

arXiv 2407.08027 ↗First posted 10 Jul 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

We introduce Fish-Visual Trait Analysis (Fish-Vista), the first organismal image dataset designed for the analysis of visual traits of aquatic species directly from images using problem formulations in computer vision. Fish-Vista contains 69,126 annotated images spanning 4,154 fish species, curated and organized to serve three downstream tasks of species classification, trait identification, and trait segmentation. Our work makes two key contributions. First, we perform a fully reproducible data processing pipeline to process images sourced from various museum collections. We annotate these images with carefully curated labels from biological databases and manual annotations to create an AI-ready dataset of visual traits, contributing to the advancement of AI in biodiversity science. Second, our proposed downstream tasks offer fertile grounds for novel computer vision research in addressing a variety of challenges such as long-tailed distributions, out-of-distribution generalization, learning with weak labels, explainable AI, and segmenting small objects. We benchmark the performance of several existing methods for our proposed tasks to expose future research opportunities in AI for biodiversity science problems involving visual traits.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 45 of its 90 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

45 of 90 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2023Segment Anythingcited by 8,158
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
33 more
  1. 2022A ConvNet for the 2020scited by 3,545
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  5. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  6. 2016Pyramid Scene Parsing Networkcited by 2,472
  7. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  8. 2019Searching for MobileNetV3cited by 1,984
  9. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  10. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  11. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  12. 2020Designing Network Design Spacescited by 954
  13. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  14. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  15. 2019Panoptic Feature Pyramid Networkscited by 644
  16. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  17. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  18. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  19. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
  20. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  21. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 159
  22. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  23. 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
  24. 2021Seeing biodiversity: perspectives in machine learning for wildlife conservationcited by 129
  25. 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 126
  26. 2020A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater Visual Analysiscited by 57
  27. 2023A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Datasetcited by 30
  28. 2021FathomNet: A global image database for enabling artificial intelligence in the oceancited by 25
  29. 2024BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversitycited by 22
  30. 2023Simplifying Neural Network Training Under Class Imbalancecited by 18
  31. 2023A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysiscited by 15
  32. 2023Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognitioncited by 11
  33. 2023Discovering Novel Biological Traits From Images Using Phylogeny-Guided Neural Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanationcited by 14
  2. 2025Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysiscited by 12
  3. 2025Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoderscited by 1
  4. 2026MATANet: A Multi-context Attention and Taxonomy-Aware Network for Fine-Grained Underwater Recognition of Marine Speciescited by 1
  5. 2025Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentationcited by 0
  6. 2025MSC: A Marine Wildlife Video Dataset with Grounded Segmentation and Clip-Level Captioningcited by 0
  7. 2026A continental-scale dataset of ground beetles with high-resolution images and validated morphological trait measurementscited by 0
  8. 2026Label-efficient underwater image classification with logistic regression on frozen foundation model embeddingscited by 0
  9. 2026Energy Constrained Hierarchical Underwater Monitoring via Local Multi-Agent RAGcited by 0