# TR-EduVSum: A Turkish-Focused Dataset and Consensus Framework for Educational Video Summarization

arXiv: https://arxiv.org/abs/2604.07553 · HTML: https://mlsanity.com/paper/2604.07553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Builds on (most cited first, 14 of 14)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation](https://mlsanity.com/paper/2212.07981/) (2022)
- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Finding a Balanced Degree of Automation for Summary Evaluation](https://mlsanity.com/paper/2109.11503/) (2021)
- [Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation](https://mlsanity.com/paper/1904.05929/) (2019)
- [QAPyramid: Fine-grained Evaluation of Content Selection for Text Summarization](https://mlsanity.com/paper/2412.07096/) (2024)
- [VT-SSum: A Benchmark Dataset for Video Transcript Segmentation and Summarization](https://mlsanity.com/paper/2106.05606/) (2021)
- [MSVD-Turkish: A Comprehensive Multimodal Dataset for Integrated Vision and Language Research in Turkish](https://mlsanity.com/paper/2012.07098/) (2020)
- [ReflectSumm: A Benchmark for Course Reflection Summarization](https://mlsanity.com/paper/2403.19012/) (2024)
- [MF2Summ: Multimodal Fusion for Video Summarization with Temporal Alignment](https://mlsanity.com/paper/2506.10430/) (2025)

## Abstract (arXiv)

This study presents a framework for generating the gold-standard summary fully automatically and reproducibly based on multiple human summaries of Turkish educational videos. Within the scope of the study, a new dataset called TR-EduVSum was created, encompassing 82 Turkish course videos in the field of "Data Structures and Algorithms" and containing a total of 3281 independent human summaries. Inspired by existing pyramid-based evaluation approaches, the AutoMUP (Automatic Meaning Unit Pyramid) method is proposed, which extracts consensus-based content from multiple human summaries. AutoMUP clusters the meaning units extracted from human summaries using embedding, statistically models inter-participant agreement, and generates graded summaries based on consensus weight. In this framework, the gold summary corresponds to the highest-consensus AutoMUP configuration, constructed from the most frequently supported meaning units across human summaries. Experimental results show that AutoMUP summaries exhibit high semantic overlap with robust LLM (Large Language Model) summaries such as Flash 2.5 and GPT-5.1. Furthermore, ablation studies clearly demonstrate the decisive role of consensus weight and clustering in determining summary quality. The proposed approach can be generalized to other Turkic languages at low cost.