# Multi-task Pairwise Neural Ranking for Hashtag Segmentation

arXiv: https://arxiv.org/abs/1906.00790 · HTML: https://mlsanity.com/paper/1906.00790/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 5 of 5)

- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Zero-shot hashtag segmentation for multilingual sentiment analysis](https://mlsanity.com/paper/2112.03213/) (2021)
- [Anti-Asian Hate Speech Detection via Data Augmented Semantic Relation Inference](https://mlsanity.com/paper/2204.07010/) (2022)
- [NTULM: Enriching Social Media Text Representations with Non-Textual Units](https://mlsanity.com/paper/2210.16586/) (2022)
- [HashSet -- A Dataset For Hashtag Segmentation](https://mlsanity.com/paper/2201.06741/) (2022)

## Abstract (arXiv)

Hashtags are often employed on social media and beyond to add metadata to a textual utterance with the goal of increasing discoverability, aiding search, or providing additional semantics. However, the semantic content of hashtags is not straightforward to infer as these represent ad-hoc conventions which frequently include multiple words joined together and can include abbreviations and unorthodox spellings. We build a dataset of 12,594 hashtags split into individual segments and propose a set of approaches for hashtag segmentation by framing it as a pairwise ranking problem between candidate segmentations. Our novel neural approaches demonstrate 24.6% error reduction in hashtag segmentation accuracy compared to the current state-of-the-art method. Finally, we demonstrate that a deeper understanding of hashtag semantics obtained through segmentation is useful for downstream applications such as sentiment analysis, for which we achieved a 2.6% increase in average recall on the SemEval 2017 sentiment analysis dataset.