mlsanityData to 7 Oct 2026

ToolFG: Towards Well-Grounded Fine-Grained Image Classification

arXiv 2606.02518 ↗First posted 1 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Fine-grained image classification (FGIC) has broad applications and has attracted significant research attention. In this paper, we explore a novel paradigm for solving FGIC by proposing ToolFG, the first tool-integrated MLLM-based framework tailored to FGIC. ToolFG enables MLLMs to autonomously and flexibly use external tools during the reasoning process, actively interact with images, and collect verifiable visual cues for distinguishing highly similar categories in a more textitreliable and textitwell-grounded manner. To equip the model with such tool-use ability, we design a novel MCTS-guided tool-use knowledge distillation mechanism, which effectively mines tool-use- and FGIC-relevant knowledge from advanced proprietary MLLMs for model training. Furthermore, we propose a model-tool co-evolution mechanism that jointly refines the toolset and the model's tool-use policy, driving them toward a mutually adapted and FGIC-specialized state. Extensive experiments demonstrate the effectiveness of our framework.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

43 of 59 references.

References · 43

  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  3. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  4. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 784
  5. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  6. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  7. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  8. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  9. 2024A Survey on Large Language Models for Code Generationcited by 646
  10. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  11. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  12. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
31 more
  1. 2015Bilinear CNNs for Fine-grained Visual Recognitioncited by 228
  2. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  3. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
  4. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  5. 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
  6. 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
  7. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 124
  8. 2021Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identificationcited by 114
  9. 2023DetGPT: Detect What You Need via Reasoningcited by 112
  10. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Enginecited by 111
  11. 2014Part-based R-CNNs for Fine-grained Category Detectioncited by 107
  12. 2018Learning to Navigate for Fine-grained Classificationcited by 103
  13. 2020Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patchescited by 92
  14. 2020Learning Attentive Pairwise Interaction for Fine-Grained Classificationcited by 84
  15. 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learningcited by 67
  16. 2019Cross-X Learning for Fine-Grained Visual Categorizationcited by 61
  17. 2017Object-Part Attention Model for Fine-grained Image Classificationcited by 54
  18. 2017Fine-graind Image Classification via Combining Vision and Languagecited by 51
  19. 2018Hierarchical Bilinear Pooling for Fine-Grained Visual Recognitioncited by 47
  20. 2020Channel Interaction Networks for Fine-Grained Image Categorizationcited by 47
  21. 2019A survey of advances in vision-based vehicle re-identificationcited by 36
  22. 2025Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Modelscited by 30
  23. 2017Fine-grained Visual-textual Representation Learningcited by 27
  24. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 23
  25. 2022SIM-Trans: Structure Information Modeling Transformer for Fine-grained Visual Categorizationcited by 19
  26. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 18
  27. 2026Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoningcited by 10
  28. 2025Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMscited by 5
  29. 2026Automatic Method Illustration Generation for AI Scientific Papers via Drawing Middleware Creation, Evolution, and Orchestrationcited by 5
  30. 2026Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Modelscited by 3
  31. 2025DiVE-k: Differential Visual Reasoning for Fine-grained Image Recognitioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Fine-Grained Image Classifications via Cascaded Vision Language Models
  2. 2024Revisiting MLLMs: An In-Depth Analysis of Image Classification Abilities
  3. 2025Vocabulary-free Fine-grained Visual Recognition via Enriched Contextually Grounded Vision-Language Model
  4. 2024Advancing Fine-Grained Classification by Structure and Subject Preserving Augmentation
  5. 2026Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: From Evaluation to Diagnosis
  6. 2024Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
  7. 2024African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification
  8. 2025Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation