mlsanityData to 7 Oct 2026

Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning

arXiv 2608.03161 ↗First posted 4 Aug 2026cs.AI cs.CLcited by 0

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Lecture videos distribute knowledge across speech, slide text, diagrams, equations, and presentation order, which transcript-only retrieval does not fully preserve. This paper presents an evidence-grounded multimodal pipeline that transcribes lectures, selects semantic anchors, applies optical character recognition (OCR), and uses a vision-language model to extract only concepts and typed relationships supported by transcript, OCR, or visual evidence. Mentions are validated and canonicalized into a provenance-rich knowledge graph. On three neural-network lectures, the pipeline processed 3,118 frames, 756 transcript segments, and 559 anchors. It retained 1,022 concept and 312 relationship mentions, yielding 172 canonical concepts and 282 relationships with 90.38% endpoint coverage. A preliminary three question retrieval test achieved 100% top-1 and top-3 accuracy and 100% mean top-5 recall. The contribution is an auditable construction method rather than a state-of-the-art performance claim.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 14 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

10 of 14 references.

References · 10

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  4. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  5. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  6. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  7. 2025LLM-empowered knowledge graph construction: A surveycited by 20
  8. 2022Kuaipedia: a Large-scale Multi-modal Short-video Encyclopediacited by 8
  9. 2025Top-Down vs. Bottom-Up Approaches for Automatic Educational Knowledge Graph Construction in CourseMappercited by 3
  10. 2025An Optimized Pipeline for Automatic Educational Knowledge Graph Constructioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026HANIA: Planner-Guided Multimodal Graph Evidence Selection for Grounded Question Answering
  2. 2026GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG
  3. 2026Q-CueGraph: Query-Conditioned Visual Evidence Graphs for Multimodal Reasoning
  4. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
  5. 2026LEMON: How Well Do MLLMs Perform Temporal Multimodal Understanding on Instructional Videos?
  6. 2026Signal or Noise? Modality Contribution and Cooperation in Multimodal GraphRAG ↗
  7. 2026ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge
  8. 2024Multimodal Reasoning with Multimodal Knowledge Graph