Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Lecture videos distribute knowledge across speech, slide text, diagrams, equations, and presentation order, which transcript-only retrieval does not fully preserve. This paper presents an evidence-grounded multimodal pipeline that transcribes lectures, selects semantic anchors, applies optical character recognition (OCR), and uses a vision-language model to extract only concepts and typed relationships supported by transcript, OCR, or visual evidence. Mentions are validated and canonicalized into a provenance-rich knowledge graph. On three neural-network lectures, the pipeline processed 3,118 frames, 756 transcript segments, and 559 anchors. It retained 1,022 concept and 312 relationship mentions, yielding 172 canonical concepts and 282 relationships with 90.38% endpoint coverage. A preliminary three question retrieval test achieved 100% top-1 and top-3 accuracy and 100% mean top-5 recall. The contribution is an auditable construction method rather than a state-of-the-art performance claim.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 14 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
10 of 14 references.
References · 10
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
- 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
- 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
- 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
- 2025LLM-empowered knowledge graph construction: A surveycited by 20
- 2022Kuaipedia: a Large-scale Multi-modal Short-video Encyclopediacited by 8
- 2025Top-Down vs. Bottom-Up Approaches for Automatic Educational Knowledge Graph Construction in CourseMappercited by 3
- 2025An Optimized Pipeline for Automatic Educational Knowledge Graph Constructioncited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026HANIA: Planner-Guided Multimodal Graph Evidence Selection for Grounded Question Answering
- 2026GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG
- 2026Q-CueGraph: Query-Conditioned Visual Evidence Graphs for Multimodal Reasoning
- 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
- 2026LEMON: How Well Do MLLMs Perform Temporal Multimodal Understanding on Instructional Videos?
- 2026Signal or Noise? Modality Contribution and Cooperation in Multimodal GraphRAG ↗
- 2026ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge
- 2024Multimodal Reasoning with Multimodal Knowledge Graph