# Attend and select: A segment selective transformer for microblog hashtag generation

arXiv: https://arxiv.org/abs/2106.03151 · HTML: https://mlsanity.com/paper/2106.03151/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Bottom-Up Abstractive Summarization](https://mlsanity.com/paper/1808.10792/) (2018)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Keyphrase Generation with Correlation Constraints](https://mlsanity.com/paper/1808.07185/) (2018)
- [Exclusive Hierarchical Decoding for Deep Keyphrase Generation](https://mlsanity.com/paper/2004.08511/) (2020)
- [Semi-Supervised Learning for Neural Keyphrase Generation](https://mlsanity.com/paper/1808.06773/) (2018)
- [Title-Guided Encoding for Keyphrase Generation](https://mlsanity.com/paper/1808.08575/) (2018)
- [Topic-Aware Neural Keyphrase Generation for Social Media Language](https://mlsanity.com/paper/1906.03889/) (2019)
- [Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention](https://mlsanity.com/paper/2008.01739/) (2020)
- [Selective Encoding for Abstractive Sentence Summarization](https://mlsanity.com/paper/1704.07073/) (2017)
- [Keyphrase Generation: A Text Summarization Struggle](https://mlsanity.com/paper/1904.00110/) (2019)
- [Microblog Hashtag Generation via Encoding Conversation Contexts](https://mlsanity.com/paper/1905.07584/) (2019)
- [On Identifying Hashtags in Disaster Twitter Data](https://mlsanity.com/paper/2001.01323/) (2020)
- [Language in Our Time: An Empirical Analysis of Hashtags](https://mlsanity.com/paper/1905.04590/) (2019)

## Built on by (most cited first, 1 of 1)

- [Hashtag-Guided Low-Resource Tweet Classification](https://mlsanity.com/paper/2302.10143/) (2023)

## Abstract (arXiv)

Hashtag generation aims to generate short and informal topical tags from a microblog post, in which tokens or phrases form the hashtags. These tokens or phrases may originate from primary fragmental textual pieces (e.g., segments) in the original text and are separated into different segments. However, conventional sequence-to-sequence generation methods are hard to filter out secondary information from different textual granularity and are not good at selecting crucial tokens. Thus, they are suboptimal in generating more condensed hashtags. In this work, we propose a modified Transformer-based generation model with adding a segments-selection procedure for the original encoding and decoding phases. The segments-selection phase is based on a novel Segments Selection Mechanism (SSM) to model different textual granularity on global text, local segments, and tokens, contributing to generating condensed hashtags. Specifically, it first attends to primary semantic segments and then transforms discontinuous segments from the source text into a sequence of hashtags by selecting crucial tokens. Extensive evaluations on the two datasets reveal our approach's superiority with significant improvements to the extraction and generation baselines. The code and datasets are available at https://github.com/OpenSUM/HashtagGen.