mlsanityData to 7 Oct 2026

LiT Tuned Models for Efficient Species Detection

arXiv 2302.10281 ↗First posted 12 Feb 2023cs.CV cs.AI cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → Agricultural vision-language models

Abstract (arXiv)

Recent advances in training vision-language models have demonstrated unprecedented robustness and transfer learning effectiveness; however, standard computer vision datasets are image-only, and therefore not well adapted to such training methods. Our paper introduces a simple methodology for adapting any fine-grained image classification dataset for distributed vision-language pretraining. We implement this methodology on the challenging iNaturalist-2021 dataset, comprised of approximately 2.7 million images of macro-organisms across 10,000 classes, and achieve a new state-of-the art model in terms of zero-shot classification accuracy. Somewhat surprisingly, our model (trained using a new method called locked-image text tuning) uses a pre-trained, frozen vision representation, proving that language alignment alone can attain strong transfer learning performance, even on fractious, long-tailed datasets. Our approach opens the door for utilizing high quality vision-language pretrained models in agriculturally relevant applications involving species detection.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agricultural vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 13 references that are in the map. First posted on arXiv 12 Feb 2023.

Builds on

10 of 13 references.

References · 10

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  4. 2021Robust fine-tuning of zero-shot modelscited by 785
  5. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  6. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
  7. 2022Class-Aware Contrastive Semi-Supervised Learningcited by 62
  8. 2021The Semi-Supervised iNaturalist Challenge at the FGVC8 Workshopcited by 16
  9. 2021Semi-Supervised Learning with Taxonomic Labelscited by 11
  10. 2022Caption supervision enables robust learnerscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Pre-training Differentially Private Models with Limited Public Datacited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2024Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions
  2. 2025Are vision-language models ready to zero-shot replace supervised classification models in agriculture?
  3. 2025Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors
  4. 2024VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images
  5. 2026Can Edge-Deployable Vision-Language Models Identify Species?
  6. 2023Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images
  7. 2026IchthyoNoma: Nomenclature and Context Sensitivity of Zero-Shot Biological Vision--Language Models for Bangladeshi Freshwater Fish Recognition
  8. 2025Vocabulary-free Fine-grained Visual Recognition via Enriched Contextually Grounded Vision-Language Model