# Reconstruct Before Summarize: An Efficient Two-Step Framework for Condensing and Summarizing Meeting Transcripts

arXiv: https://arxiv.org/abs/2305.07988 · HTML: https://mlsanity.com/paper/2305.07988/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 13 May 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [Sparse Sinkhorn Attention](https://mlsanity.com/paper/2002.11296/) (2020)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Structured Attention Networks](https://mlsanity.com/paper/1702.00887/) (2017)
- [DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization](https://mlsanity.com/paper/2109.02492/) (2021)
- [Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents](https://mlsanity.com/paper/2110.10150/) (2021)
- [Investigating the influence of noise and distractors on the interpretation of neural networks](https://mlsanity.com/paper/1611.07270/) (2016)
- [Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization](https://mlsanity.com/paper/1805.05271/) (2018)
- [Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization](https://mlsanity.com/paper/2105.12544/) (2021)
- [Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts](https://mlsanity.com/paper/1809.05715/) (2018)
- [Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversations](https://mlsanity.com/paper/1910.01335/) (2019)
- [Coreference-Aware Dialogue Summarization](https://mlsanity.com/paper/2106.08556/) (2021)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [Dialogue Discourse-Aware Graph Model and Data Augmentation for Meeting Summarization](https://mlsanity.com/paper/2012.03502/) (2020)
- [An Empirical Study on Explanations in Out-of-Domain Settings](https://mlsanity.com/paper/2203.00056/) (2022)

## Abstract (arXiv)

Meetings typically involve multiple participants and lengthy conversations, resulting in redundant and trivial content. To overcome these challenges, we propose a two-step framework, Reconstruct before Summarize (RbS), for effective and efficient meeting summarization. RbS first leverages a self-supervised paradigm to annotate essential contents by reconstructing the meeting transcripts. Secondly, we propose a relative positional bucketing (RPB) algorithm to equip (conventional) summarization models to generate the summary. Despite the additional reconstruction process, our proposed RPB significantly compressed the input, leading to faster processing and reduced memory consumption compared to traditional summarization methods. We validate the effectiveness and efficiency of our method through extensive evaluations and analysis. On two meeting summarization datasets, AMI and ICSI, our approach outperforms previous state-of-the-art approaches without relying on large-scale pre-training or expert-grade annotating tools.