# Gemini Pro Defeated by GPT-4V: Evidence from Education

arXiv: https://arxiv.org/abs/2401.08660 · HTML: https://mlsanity.com/paper/2401.08660/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Fast Transformer Decoding: One Write-Head is All You Need](https://mlsanity.com/paper/1911.02150/) (2019)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [Applying Large Language Models and Chain-of-Thought for Automatic Scoring](https://mlsanity.com/paper/2312.03748/) (2023)
- [A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise](https://mlsanity.com/paper/2312.12436/) (2023)
- [An In-depth Look at Gemini's Language Abilities](https://mlsanity.com/paper/2312.11444/) (2023)
- [Using GPT-4 to Augment Unbalanced Data for Automatic Scoring](https://mlsanity.com/paper/2310.18365/) (2023)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [NERIF: GPT-4V for Automatic Scoring of Drawn Models](https://mlsanity.com/paper/2311.12990/) (2023)
- [Fully Authentic Visual Question Answering Dataset from Online Communities](https://mlsanity.com/paper/2311.15562/) (2023)
- [AI Gender Bias, Disparities, and Fairness: Does Training Data Matter?](https://mlsanity.com/paper/2312.10833/) (2023)
- [GPT-4V(ision) Unsuitable for Clinical Care and Education: A Clinician-Evaluated Assessment](https://mlsanity.com/paper/2403.12046/) (2023)
- [An Evaluation of GPT-4V and Gemini in Online VQA](https://mlsanity.com/paper/2312.10637/) (2023)

## Built on by (most cited first, 12 of 12)

- [HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs](https://mlsanity.com/paper/2406.06027/) (2024)
- [Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink](https://mlsanity.com/paper/2501.15269/) (2025)
- [Transforming Teachers' Roles and Agencies in the Era of Generative AI: Perceptions, Acceptance, Knowledge, and Practices](https://mlsanity.com/paper/2410.03018/) (2024)
- [SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches](https://mlsanity.com/paper/2507.22904/) (2025)
- [G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in German](https://mlsanity.com/paper/2402.06584/) (2024)
- [Fine-tuning and Prompt Engineering with Cognitive Knowledge Graphs for Scholarly Knowledge Organization](https://mlsanity.com/paper/2409.06433/) (2024)
- [An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analyses](https://mlsanity.com/paper/2503.21393/) (2025)
- [GPT-4V Explorations: Mining Autonomous Driving](https://mlsanity.com/paper/2406.16817/) (2024)
- [DeCoT: Decomposing Complex Instructions for Enhanced Text-to-Image Generation with Large Language Models](https://mlsanity.com/paper/2508.12396/) (2025)
- [Multi-method Integration with Confidence-based Weighting for Zero-shot Image Classification](https://mlsanity.com/paper/2405.02155/) (2024)
- [Generative Artificial Intelligence-Guided User Studies: An Application for Air Taxi Services](https://mlsanity.com/paper/2406.12296/) (2024)
- [Captioning Daily Activity Images in Early Childhood Education: Benchmark and Algorithm](https://mlsanity.com/paper/2604.01941/) (2026)

## Abstract (arXiv)

This study compared the classification performance of Gemini Pro and GPT-4V in educational settings. Employing visual question answering (VQA) techniques, the study examined both models' abilities to read text-based rubrics and then automatically score student-drawn models in science education. We employed both quantitative and qualitative analyses using a dataset derived from student-drawn scientific models and employing NERIF (Notation-Enhanced Rubrics for Image Feedback) prompting methods. The findings reveal that GPT-4V significantly outperforms Gemini Pro in terms of scoring accuracy and Quadratic Weighted Kappa. The qualitative analysis reveals that the differences may be due to the models' ability to process fine-grained texts in images and overall image classification performance. Even adapting the NERIF approach by further de-sizing the input images, Gemini Pro seems not able to perform as well as GPT-4V. The findings suggest GPT-4V's superior capability in handling complex multimodal educational tasks. The study concludes that while both models represent advancements in AI, GPT-4V's higher performance makes it a more suitable tool for educational applications involving multimodal data interpretation.