# Chain of Visual Perception: Harnessing Multimodal Large Language Models for Zero-shot Camouflaged Object Detection

arXiv: https://arxiv.org/abs/2311.11273 · HTML: https://mlsanity.com/paper/2311.11273/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic](https://mlsanity.com/paper/2306.15195/) (2023)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest](https://mlsanity.com/paper/2307.03601/) (2023)
- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [Anabranch Network for Camouflaged Object Segmentation](https://mlsanity.com/paper/2105.09451/) (2021)
- [Self-supervised Video Object Segmentation by Motion Grouping](https://mlsanity.com/paper/2104.07658/) (2021)
- [Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching](https://mlsanity.com/paper/2305.13310/) (2023)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [DetGPT: Detect What You Need via Reasoning](https://mlsanity.com/paper/2305.14167/) (2023)
- [Mutual Graph Learning for Camouflaged Object Detection](https://mlsanity.com/paper/2104.02613/) (2021)
- [Feature Shrinkage Pyramid for Camouflaged Object Detection with Transformers](https://mlsanity.com/paper/2303.14816/) (2023)
- [Deep Gradient Learning for Efficient Camouflaged Object Detection](https://mlsanity.com/paper/2205.12853/) (2022)
- [ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuning](https://mlsanity.com/paper/2307.09474/) (2023)
- [Weakly-Supervised Camouflaged Object Detection with Scribble Annotations](https://mlsanity.com/paper/2207.14083/) (2022)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [Fast Camouflaged Object Detection via Edge-based Reversible Re-calibration Network](https://mlsanity.com/paper/2111.03216/) (2021)
- [Towards Generic Anomaly Detection and Understanding: Large-scale Visual-linguistic Model (GPT-4V) Takes the Lead](https://mlsanity.com/paper/2311.02782/) (2023)
- [Camouflaged Object Detection and Tracking: A Survey](https://mlsanity.com/paper/2012.13581/) (2020)

## Built on by (most cited first, 22 of 22)

- [A Survey of Camouflaged Object Detection and Beyond](https://mlsanity.com/paper/2408.14562/) (2024)
- [SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning](https://mlsanity.com/paper/2505.22596/) (2025)
- [Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2406.16449/) (2024)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [Evaluating SAM2's Role in Camouflaged Object Detection: From SAM to SAM2](https://mlsanity.com/paper/2407.21596/) (2024)
- [When SAM2 Meets Video Camouflaged Object Segmentation: A Comprehensive Evaluation and Adaptation](https://mlsanity.com/paper/2409.18653/) (2024)
- [Deep Learning in Concealed Dense Prediction](https://mlsanity.com/paper/2504.10979/) (2025)
- [Beyond Single Images: Retrieval Self-Augmented Unsupervised Camouflaged Object Detection](https://mlsanity.com/paper/2510.18437/) (2025)
- [Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Models](https://mlsanity.com/paper/2603.16253/) (2026)
- [Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations](https://mlsanity.com/paper/2410.16953/) (2024)
- [LLM-EvRep: Learning an LLM-Compatible Event Representation Using a Self-Supervised Framework](https://mlsanity.com/paper/2502.14273/) (2025)
- [Align and Surpass Human Camouflaged Perception: Visual Refocus Reinforcement Fine-Tuning](https://mlsanity.com/paper/2505.19611/) (2025)
- [Stepwise Decomposition and Dual-stream Focus: A Novel Approach for Training-free Camouflaged Object Segmentation](https://mlsanity.com/paper/2506.06818/) (2025)
- [SAEC: Scene-Aware Enhanced Edge-Cloud Collaborative Industrial Vision Inspection with Multimodal LLM](https://mlsanity.com/paper/2509.17136/) (2025)
- [Discover, Segment, and Select: A Progressive Mechanism for Zero-shot Camouflaged Object Segmentation](https://mlsanity.com/paper/2602.19944/) (2026)
- [Marine Saliency Segmenter: Object-Focused Conditional Diffusion with Region-Level Semantic Knowledge Distillation](https://mlsanity.com/paper/2504.02391/) (2025)
- [ZS-VCOS: Zero-Shot Video Camouflaged Object Segmentation By Optical Flow and Open Vocabulary Object Detection](https://mlsanity.com/paper/2505.01431/) (2025)
- [AutoMiSeg: Automatic Medical Image Segmentation via Test-Time Adaptation of Foundation Models](https://mlsanity.com/paper/2505.17931/) (2025)
- [Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM](https://mlsanity.com/paper/2509.06422/) (2025)
- [Classifier-Centric Adaptive Framework for Open-Vocabulary Camouflaged Object Segmentation](https://mlsanity.com/paper/2509.24681/) (2025)
- [Camouflage-aware Image-Text Retrieval via Expert Collaboration](https://mlsanity.com/paper/2604.01251/) (2026)
- [AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture](https://mlsanity.com/paper/2605.22366/) (2026)

## Abstract (arXiv)

In this paper, we introduce a novel multimodal camo-perceptive framework (MMCPF) aimed at handling zero-shot Camouflaged Object Detection (COD) by leveraging the powerful capabilities of Multimodal Large Language Models (MLLMs). Recognizing the inherent limitations of current COD methodologies, which predominantly rely on supervised learning models demanding extensive and accurately annotated datasets, resulting in weak generalization, our research proposes a zero-shot MMCPF that circumvents these challenges. Although MLLMs hold significant potential for broad applications, their effectiveness in COD is hindered and they would make misinterpretations of camouflaged objects. To address this challenge, we further propose a strategic enhancement called the Chain of Visual Perception (CoVP), which significantly improves the perceptual capabilities of MLLMs in camouflaged scenes by leveraging both linguistic and visual cues more effectively. We validate the effectiveness of MMCPF on five widely used COD datasets, containing CAMO, COD10K, NC4K, MoCA-Mask and OVCamo. Experiments show that MMCPF can outperform all existing state-of-the-art zero-shot COD methods, and achieve competitive performance compared to weakly-supervised and fully-supervised methods, which demonstrates the potential of MMCPF. The Github link of this paper is urlhttps://github.com/luckybird1994/MMCPF.