# State Space Models for Extractive Summarization in Low Resource Scenarios

arXiv: https://arxiv.org/abs/2501.14673 · HTML: https://mlsanity.com/paper/2501.14673/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 24 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Unsupervised Opinion Summarization as Copycat-Review Generation](https://mlsanity.com/paper/1911.02247/) (2019)
- [Neural Document Summarization by Jointly Learning to Score and Select Sentences](https://mlsanity.com/paper/1807.02305/) (2018)
- [AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization](https://mlsanity.com/paper/2103.11332/) (2021)
- [Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks](https://mlsanity.com/paper/1908.10423/) (2019)
- [Long Document Summarization in a Low Resource Setting using Pretrained Language Models](https://mlsanity.com/paper/2103.00751/) (2021)
- [Stepwise Extractive Summarization and Planning with Structured Transformers](https://mlsanity.com/paper/2010.02744/) (2020)
- [Adaptive Poincar'e Point to Set Distance for Few-Shot Classification](https://mlsanity.com/paper/2112.01719/) (2021)
- [Meta-Transfer Learning for Low-Resource Abstractive Summarization](https://mlsanity.com/paper/2102.09397/) (2021)
- [GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization](https://mlsanity.com/paper/2208.09982/) (2022)
- [On Optimal Transformer Depth for Low-Resource Language Translation](https://mlsanity.com/paper/2004.04418/) (2020)
- [Parameter-Efficient Language Model Tuning with Active Learning in Low-Resource Settings](https://mlsanity.com/paper/2305.14576/) (2023)
- [Fast and Simple Spectral Clustering in Theory and Practice](https://mlsanity.com/paper/2310.10939/) (2023)
- [On the Choice of Auxiliary Languages for Improved Sequence Tagging](https://mlsanity.com/paper/2005.09389/) (2020)

## Abstract (arXiv)

Extractive summarization involves selecting the most relevant sentences from a text. Recently, researchers have focused on advancing methods to improve state-of-the-art results in low-resource settings. Motivated by these advancements, we propose the MPoincareSum method. This method applies the Mamba state space model to generate the semantics of reviews and sentences, which are then concatenated. A Poincare compression is used to select the most meaningful features, followed by the application of a linear layer to predict sentence relevance based on the corresponding review. Finally, we paraphrase the relevant sentences to create the final summary. To evaluate the effectiveness of MPoincareSum, we conducted extensive experiments using the Amazon review dataset. The performance of the method was assessed using ROUGE scores. The experimental results demonstrate that MPoincareSum outperforms several existing approaches in the literature