# Simulating Validity: Modal Decoupling in MLLM Generated Feedback on Science Drawings

arXiv: https://arxiv.org/abs/2604.26957 · HTML: https://mlsanity.com/paper/2604.26957/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 5 Apr 2026.

## Builds on (most cited first, 11 of 11)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [Multi-Modal Hallucination Control by Visual Information Grounding](https://mlsanity.com/paper/2403.14003/) (2024)
- [MLLMs are Deeply Affected by Modality Bias](https://mlsanity.com/paper/2505.18657/) (2025)
- [List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs](https://mlsanity.com/paper/2404.16375/) (2024)
- [v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning](https://mlsanity.com/paper/2505.18842/) (2025)
- [Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education](https://mlsanity.com/paper/2401.00832/) (2024)
- [Decipher the Modality Gap in Multimodal Contrastive Learning: From Convergent Representations to Pairwise Alignment](https://mlsanity.com/paper/2510.03268/) (2025)
- [SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches](https://mlsanity.com/paper/2507.22904/) (2025)

## Abstract (arXiv)

In science education, students frequently construct hand-drawn visual models of scientific phenomena. These drawings rely on a visual structure where information is encoded through visual objects, their attributes, and relationships. Multimodal large language models (MLLMs) are increasingly used to generate feedback on students' hand-drawn scientific models. However, the validity of such feedback depends on whether model claims are grounded in the specific visual evidence of the student drawing. This study uncovers grounding failures, consistent with modal decoupling, in off-the-shelf MLLM feedback, where outputs remain pedagogically plausible in form while contradicting the drawing or treating depicted elements as missing. Using N = 150 middle school drawings from a kinetic molecular theory unit spanning five modeling tasks and three competence levels, we generated N = 300 feedback instances with GPT-5.1. All outputs were coded for four grounding error types: object mismatch, attribute mismatch, relation mismatch, and false absence. Grounding failures were common: 41.3% of feedback instances contained at least one error. An inventory-list-first workflow reduced several error categories and lowered the overall error rate, but it did not resolve the underlying limitation: approximately one in three outputs remained flawed, with false absence as the dominant failure mode. Moreover, feedback that appears visually grounded offered little diagnostic value for identifying invalid instances. The findings indicate that modal decoupling is a substantial limitation and that valid feedback will require grounding mechanisms beyond common prompting strategies.