# How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges

arXiv: https://arxiv.org/abs/2307.15016 · HTML: https://mlsanity.com/paper/2307.15016/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)
- [Video Polyp Segmentation: A Deep Learning Perspective](https://mlsanity.com/paper/2203.14291/) (2022)
- [Advances in Deep Concealed Scene Understanding](https://mlsanity.com/paper/2304.11234/) (2023)
- [Visual Sentiment Analysis from Disaster Images in Social Media](https://mlsanity.com/paper/2009.03051/) (2020)
- [RGB-D Indiscernible Object Counting in Underwater Scenes](https://mlsanity.com/paper/2304.11677/) (2023)

## Built on by (most cited first, 9 of 9)

- [Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning](https://mlsanity.com/paper/2411.18203/) (2024)
- [Superposition Yields Robust Neural Scaling](https://mlsanity.com/paper/2505.10465/) (2025)
- [Effectiveness Assessment of Recent Large Vision-Language Models](https://mlsanity.com/paper/2403.04306/) (2024)
- [GlitchBench: Can large multimodal models detect video game glitches?](https://mlsanity.com/paper/2312.05291/) (2023)
- [FOCUS: First Order Concentrated Updating Scheme](https://mlsanity.com/paper/2501.12243/) (2025)
- [Give me a hint: Can LLMs take a hint to solve math problems?](https://mlsanity.com/paper/2410.05915/) (2024)
- [Multimodal Analysis Of Google Bard And GPT-Vision: Experiments In Visual Reasoning](https://mlsanity.com/paper/2309.16705/) (2023)
- [Patch is Enough: Naturalistic Adversarial Patch against Vision-Language Pre-training Models](https://mlsanity.com/paper/2410.04884/) (2024)
- [VERITAS: Verification and Explanation of Realness in Images for Transparency in AI Systems](https://mlsanity.com/paper/2507.05146/) (2025)

## Abstract (arXiv)

Google's Bard has emerged as a formidable competitor to OpenAI's ChatGPT in the field of conversational AI. Notably, Bard has recently been updated to handle visual inputs alongside text prompts during conversations. Given Bard's impressive track record in handling textual inputs, we explore its capabilities in understanding and interpreting visual data (images) conditioned by text questions. This exploration holds the potential to unveil new insights and challenges for Bard and other forthcoming multi-modal Generative models, especially in addressing complex computer vision problems that demand accurate visual and language understanding. Specifically, in this study, we focus on 15 diverse task scenarios encompassing regular, camouflaged, medical, under-water and remote sensing data to comprehensively evaluate Bard's performance. Our primary finding indicates that Bard still struggles in these vision scenarios, highlighting the significant gap in vision-based understanding that needs to be bridged in future developments. We expect that this empirical study will prove valuable in advancing future models, leading to enhanced capabilities in comprehending and interpreting fine-grained visual data. Our project is released on https://github.com/htqin/GoogleBard-VisUnderstand