# ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning

arXiv: https://arxiv.org/abs/2408.02210 · HTML: https://mlsanity.com/paper/2408.02210/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 3 of its 78 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 3 of 3)

- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [Prismer: A Vision-Language Model with Multi-Task Experts](https://mlsanity.com/paper/2303.02506/) (2023)

## Built on by (most cited first, 6 of 6)

- [VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridges](https://mlsanity.com/paper/2409.01071/) (2024)
- [LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception](https://mlsanity.com/paper/2504.15362/) (2025)
- [ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection](https://mlsanity.com/paper/2505.16475/) (2025)
- [A Survey on Feedback-based Multi-step Reasoning for Large Language Models on Mathematics](https://mlsanity.com/paper/2502.14333/) (2025)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [Efficiently Enhancing General Agents With Hierarchical-categorical Memory](https://mlsanity.com/paper/2505.22006/) (2025)

## Abstract (arXiv)

Compositional visual reasoning methods, which translate a complex query into a structured composition of feasible visual tasks, have exhibited a strong potential in complicated multi-modal tasks. Empowered by recent advances in large language models (LLMs), this multi-modal challenge has been brought to a new stage by treating LLMs as few-shot/zero-shot planners, i.e., vision-language (VL) programming. Such methods, despite their numerous merits, suffer from challenges due to LLM planning mistakes or inaccuracy of visual execution modules, lagging behind the non-compositional models. In this work, we devise a "plug-and-play" method, ExoViP, to correct errors in both the planning and execution stages through introspective verification. We employ verification modules as "exoskeletons" to enhance current VL programming schemes. Specifically, our proposed verification module utilizes a mixture of three sub-verifiers to validate predictions after each reasoning step, subsequently calibrating the visual module predictions and refining the reasoning trace planned by LLMs. Experimental results on two representative VL programming methods showcase consistent improvements on five compositional reasoning tasks on standard benchmarks. In light of this, we believe that ExoViP can foster better performance and generalization on open-domain multi-modal challenges.