mlsanityData to 7 Oct 2026

Revisiting KRISP: A Lightweight Reproduction and Analysis of Knowledge-Enhanced Vision-Language Models

arXiv 2511.20795 ↗First posted 25 Nov 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Facebook AI Research introduced KRISP [4], which integrates structured external knowledge into pipelines for vision-language reasoning. Despite its effectiveness, the original model has been developed for industrial-scale training, is computationally demanding, and is tightly connected to a large backbone. In this work, we reexamine KRISP from a different angle and offer a lightweight reproduction with significantly fewer parameters. Even though our replicated model performs about 75 % of the original, the replication process uncovers a number of design flaws, real-world pitfalls, and implicit problems that were not fully covered in the original paper. We offer insights into the scalability and efficacy of knowledge-enhanced VQA architectures under resource constraints through systematic ablation studies, which include a proof-of-concept on synthetic VQA data and evaluation on the DAQUAR dataset. Our model, configured with a low parameter setup and constrained by the external Knowledge graph domain, prevents AI hallucinations and generates outputs solely within that domain. Minimal parameters allow us to function on edge devices like smartphones and AR-VR, further improving offline visual reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 19 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

9 of 19 references.

References · 9

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  6. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  7. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  8. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  9. 2014A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Inputcited by 112

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Knowledge Integration for Enhanced Vision-Language Reasoning
  2. 2023Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models
  3. 2024UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
  4. 2026Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
  5. 2026SmoGVLM: A Small, Graph-enhanced Vision-Language Model
  6. 2025A Comprehensive Survey of Knowledge-Based Vision Question Answering Systems: The Lifecycle of Knowledge in Visual Reasoning Task
  7. 2024VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models
  8. 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs