# FungalZSL: Zero-Shot Fungal Classification with Image Captioning Using a Synthetic Data Approach

arXiv: https://arxiv.org/abs/2502.19038 · HTML: https://mlsanity.com/paper/2502.19038/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agricultural vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Agricultural vision-language models](https://mlsanity.com/mllm/agricultural-vision-language-models/)

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets](https://mlsanity.com/paper/2302.02551/) (2023)
- [CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data](https://mlsanity.com/paper/2404.15653/) (2024)
- [Synthetic Fungi Datasets: A Time-Aligned Approach](https://mlsanity.com/paper/2501.02855/) (2025)

## Built on by (most cited first, 1 of 1)

- [CLIPTime: Time-Aware Multimodal Representation Learning from Images and Text](https://mlsanity.com/paper/2508.00447/) (2025)

## Abstract (arXiv)

The effectiveness of zero-shot classification in large vision-language models (VLMs), such as Contrastive Language-Image Pre-training (CLIP), depends on access to extensive, well-aligned text-image datasets. In this work, we introduce two complementary data sources, one generated by large language models (LLMs) to describe the stages of fungal growth and another comprising a diverse set of synthetic fungi images. These datasets are designed to enhance CLIPs zero-shot classification capabilities for fungi-related tasks. To ensure effective alignment between text and image data, we project them into CLIPs shared representation space, focusing on different fungal growth stages. We generate text using LLaMA3.2 to bridge modality gaps and synthetically create fungi images. Furthermore, we investigate knowledge transfer by comparing text outputs from different LLM techniques to refine classification across growth stages.