# SimCSum: Joint Learning of Simplification and Cross-lingual Summarization for Cross-lingual Science Journalism

arXiv: https://arxiv.org/abs/2304.01621 · HTML: https://mlsanity.com/paper/2304.01621/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization](https://mlsanity.com/paper/2010.03093/) (2020)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [Better Summarization Evaluation with Word Embeddings for ROUGE](https://mlsanity.com/paper/1508.06034/) (2015)
- [NCLS: Neural Cross-Lingual Summarization](https://mlsanity.com/paper/1909.00156/) (2019)
- [Making Science Simple: Corpora for the Lay Summarisation of Scientific Literature](https://mlsanity.com/paper/2210.09932/) (2022)
- [Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text](https://mlsanity.com/paper/2107.03444/) (2021)
- [Text Simplification by Tagging](https://mlsanity.com/paper/2103.05070/) (2021)
- [Cross-Lingual Abstractive Summarization with Limited Parallel Resources](https://mlsanity.com/paper/2105.13648/) (2021)
- [Document-Level Text Simplification: Dataset, Criteria and Baseline](https://mlsanity.com/paper/2110.05071/) (2021)
- [Mixed-Lingual Pre-training for Cross-lingual Summarization](https://mlsanity.com/paper/2010.08892/) (2020)
- [Klexikon: A German Dataset for Joint Summarization and Simplification](https://mlsanity.com/paper/2201.07198/) (2022)
- [X-SCITLDR: Cross-Lingual Extreme Summarization of Scholarly Documents](https://mlsanity.com/paper/2205.15051/) (2022)
- [Data-driven Summarization of Scientific Articles](https://mlsanity.com/paper/1804.08875/) (2018)
- [Multi-Task Learning for Cross-Lingual Abstractive Summarization](https://mlsanity.com/paper/2010.07503/) (2020)
- [Unifying Cross-lingual Summarization and Machine Translation with Compression Rate](https://mlsanity.com/paper/2110.07936/) (2021)
- [Towards Abstraction from Extraction: Multiple Timescale Gated Recurrent Unit for Summarization](https://mlsanity.com/paper/1607.00718/) (2016)
- [CUT: Controllable Unsupervised Text Simplification](https://mlsanity.com/paper/2012.01936/) (2020)

## Abstract (arXiv)

Cross-lingual science journalism generates popular science stories of scientific articles different from the source language for a non-expert audience. Hence, a cross-lingual popular summary must contain the salient content of the input document, and the content should be coherent, comprehensible, and in a local language for the targeted audience. We improve these aspects of cross-lingual summary generation by joint training of two high-level NLP tasks, simplification and cross-lingual summarization. The former task reduces linguistic complexity, and the latter focuses on cross-lingual abstractive summarization. We propose a novel multi-task architecture - SimCSum consisting of one shared encoder and two parallel decoders jointly learning simplification and cross-lingual summarization. We empirically investigate the performance of SimCSum by comparing it with several strong baselines over several evaluation metrics and by human evaluation. Overall, SimCSum demonstrates statistically significant improvements over the state-of-the-art on two non-synthetic cross-lingual scientific datasets. Furthermore, we conduct an in-depth investigation into the linguistic properties of generated summaries and an error analysis.