# A Framework for Fast Polarity Labelling of Massive Data Streams

arXiv: https://arxiv.org/abs/2203.12368 · HTML: https://mlsanity.com/paper/2203.12368/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 23 Mar 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Deep Short Text Classification with Knowledge Powered Attention](https://mlsanity.com/paper/1902.08050/) (2019)
- [Lambda Learner: Fast Incremental Learning on Data Streams](https://mlsanity.com/paper/2010.05154/) (2020)
- [Distributed Real-Time Sentiment Analysis for Big Data Social Streams](https://mlsanity.com/paper/1612.08543/) (2016)
- [Towards Concurrent Stateful Stream Processing on Multicore Processors (Technical Report)](https://mlsanity.com/paper/1904.03800/) (2019)

## Abstract (arXiv)

Many of the existing sentiment analysis techniques are based on supervised learning, and they demand the availability of valuable training datasets to train their models. When dataset freshness is critical, the annotating of high speed unlabelled data streams becomes critical but remains an open problem. In this paper, we propose PLStream, a novel Apache Flink-based framework for fast polarity labelling of massive data streams, like Twitter tweets or online product reviews. We address the associated implementation challenges and propose a list of techniques including both algorithmic improvements and system optimizations. A thorough empirical validation with two real-world workloads demonstrates that PLStream is able to generate high quality labels (almost 80% accuracy) in the presence of high-speed continuous unlabelled data streams (almost 16,000 tuples/sec) without any manual efforts.