# PORTRAIT: a hybrid aPproach tO cReate extractive ground-TRuth summAry for dIsaster evenT arXiv: https://arxiv.org/abs/2305.11536 ยท HTML: https://mlsanity.com/paper/2305.11536/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 26 of its 90 references that are in the map. - First posted on arXiv 19 May 2023. ## Builds on (most cited first, 26 of 26) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018) - [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019) - [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019) - [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019) - [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021) - [DialogSum: A Real-Life Scenario Dialogue Summarization Dataset](https://mlsanity.com/paper/2105.06762/) (2021) - [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018) - [SummScreen: A Dataset for Abstractive Screenplay Summarization](https://mlsanity.com/paper/2104.07091/) (2021) - [ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks](https://mlsanity.com/paper/1909.01716/) (2019) - [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://mlsanity.com/paper/2304.04193/) (2023) - [Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages](https://mlsanity.com/paper/1605.05894/) (2016) - [HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks](https://mlsanity.com/paper/2104.03090/) (2021) - [TWEETSUMM -- A Dialog Summarization Dataset for Customer Service](https://mlsanity.com/paper/2111.11894/) (2021) - [ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization](https://mlsanity.com/paper/2202.05599/) (2022) - [Automatic Text Summarization Methods: A Comprehensive Review](https://mlsanity.com/paper/2204.01849/) (2022) - [CAVES: A Dataset to facilitate Explainable Classification and Summarization of Concerns towards COVID Vaccines](https://mlsanity.com/paper/2204.13746/) (2022) - [Unsupervised Detection of Sub-events in Large Scale Disasters](https://mlsanity.com/paper/1912.13332/) (2019) - [GameWikiSum: a Novel Large Multi-Document Summarization Dataset](https://mlsanity.com/paper/2002.06851/) (2020) - [Utilizing Microblogs for Assisting Post-Disaster Relief Operations via Matching Resource Needs and Availabilities](https://mlsanity.com/paper/2007.09368/) (2020) - [empathi: An ontology for Emergency Managing and Planning about Hazard Crisis](https://mlsanity.com/paper/1810.12510/) (2018) - [TGSum: Build Tweet Guided Multi-Document Summarization Dataset](https://mlsanity.com/paper/1511.08417/) (2015) - [OntoDSumm : Ontology based Tweet Summarization for Disaster Events](https://mlsanity.com/paper/2201.06545/) (2022) - [EnDSUM: Entropy and Diversity based Disaster Tweet Summarization](https://mlsanity.com/paper/2203.01188/) (2022) - [TSSuBERT: Tweet Stream Summarization Using BERT](https://mlsanity.com/paper/2106.08770/) (2021) - [Summarizing Situational and Topical Information During Crises](https://mlsanity.com/paper/1610.01561/) (2016) ## Built on by (most cited first, 3 of 3) - [ATSumm: Auxiliary information enhanced approach for abstractive disaster Tweet Summarization with sparse training data](https://mlsanity.com/paper/2405.06541/) (2024) - [ADSumm: Annotated Ground-truth Summary Datasets for Disaster Tweet Summarization](https://mlsanity.com/paper/2405.06551/) (2024) - [EquiSumm : A Gender Bias-Aware Framework for Inclusive Tweet Summarization](https://mlsanity.com/paper/2605.23412/) (2026) ## Abstract (arXiv) Disaster summarization approaches provide an overview of the important information posted during disaster events on social media platforms, such as, Twitter. However, the type of information posted significantly varies across disasters depending on several factors like the location, type, severity, etc. Verification of the effectiveness of disaster summarization approaches still suffer due to the lack of availability of good spectrum of datasets along with the ground-truth summary. Existing approaches for ground-truth summary generation (ground-truth for extractive summarization) relies on the wisdom and intuition of the annotators. Annotators are provided with a complete set of input tweets from which a subset of tweets is selected by the annotators for the summary. This process requires immense human effort and significant time. Additionally, this intuition-based selection of the tweets might lead to a high variance in summaries generated across annotators. Therefore, to handle these challenges, we propose a hybrid (semi-automated) approach (PORTRAIT) where we partly automate the ground-truth summary generation procedure. This approach reduces the effort and time of the annotators while ensuring the quality of the created ground-truth summary. We validate the effectiveness of PORTRAIT on 5 disaster events through quantitative and qualitative comparisons of ground-truth summaries generated by existing intuitive approaches, a semi-automated approach, and PORTRAIT. We prepare and release the ground-truth summaries for 5 disaster events which consist of both natural and man-made disaster events belonging to 4 different countries. Finally, we provide a study about the performance of various state-of-the-art summarization approaches on the ground-truth summaries generated by PORTRAIT using ROUGE-N F1-scores.