# PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction

arXiv: https://arxiv.org/abs/2604.17278 · HTML: https://mlsanity.com/paper/2604.17278/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [UniFormer: Unifying Convolution and Self-attention for Visual Recognition](https://mlsanity.com/paper/2201.09450/) (2022)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures](https://mlsanity.com/paper/2403.02308/) (2024)
- [Restore-RWKV: Efficient and Effective Medical Image Restoration with RWKV](https://mlsanity.com/paper/2407.11087/) (2024)
- [Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding](https://mlsanity.com/paper/2311.15206/) (2023)
- [BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity](https://mlsanity.com/paper/2406.12723/) (2024)
- [RWKV-CLIP: A Robust Vision-Language Representation Learner](https://mlsanity.com/paper/2406.06973/) (2024)
- [TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition](https://mlsanity.com/paper/2310.19380/) (2023)
- [SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognition](https://mlsanity.com/paper/2301.13156/) (2023)
- [Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Model](https://mlsanity.com/paper/2406.19369/) (2024)
- [QuadMamba: Learning Quadtree-based Selective Scan for Visual State Space Model](https://mlsanity.com/paper/2410.06806/) (2024)
- [Exploring Real&Synthetic Dataset and Linear Attention in Image Restoration](https://mlsanity.com/paper/2412.03814/) (2024)
- [StyleRWKV: High-Quality and High-Efficiency Style Transfer with RWKV-like Architecture](https://mlsanity.com/paper/2412.19535/) (2024)

## Abstract (arXiv)

Effective pest recognition and management are crucial for sustainable agricultural development. However, collecting pest data in real scenarios is often challenging. Compared to other domains, pests exhibit a wide variety of species with complex and diverse morphological characteristics. Existing techniques struggle to effectively model the key visual and high-level semantic features of pests in a fine-grained manner. These limitations hinder the practical application of such methods in real agricultural scenarios. To address these critical challenges, we present a synergistic approach that integrates PestVL-Net, a novel vision-language framework, with two multi-species pest datasets to facilitate fine-grained pest learning. The visual pathway of PestVL-Net utilizes the Recurrent Weighted Key Value (RWKV) architecture, incorporating a saliency-guided adaptive window partitioning scheme to effectively model the fine-grained visual characteristics of pests. Concurrently, the linguistic component generates precise pest semantic descriptions by leveraging Multimodal Large Language Models (MLLMs) priors, critically informed by agricultural expert knowledge and structured via multimodal Chain-of-Thought (CoT) reasoning. The deep fusion of these complementary visual and textual representations enables fine-grained multimodal pest learning. Extensive experimental evaluations on multiple pest datasets validate the superior performance of PestVL-Net, highlighting its potential for effective real-world pest management.