# Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation

arXiv: https://arxiv.org/abs/2103.06490 · HTML: https://mlsanity.com/paper/2103.06490/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 54 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [Deep Active Learning for Named Entity Recognition](https://mlsanity.com/paper/1707.05928/) (2017)
- [Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study](https://mlsanity.com/paper/1808.05697/) (2018)
- [When is multitask learning effective? Semantic sequence prediction under varying data conditions](https://mlsanity.com/paper/1612.02251/) (2016)
- [Active Learning for Interactive Neural Machine Translation of Data Streams](https://mlsanity.com/paper/1807.11243/) (2018)
- [Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation](https://mlsanity.com/paper/1410.5877/) (2014)

## Built on by (most cited first, 6 of 6)

- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Onception: Active Learning with Expert Advice for Real World Machine Translation](https://mlsanity.com/paper/2203.04507/) (2022)
- [Towards Selection of Text-to-speech Data to Augment ASR Training](https://mlsanity.com/paper/2306.00998/) (2023)
- [Impact of Stop Sets on Stopping Active Learning for Text Classification](https://mlsanity.com/paper/2201.05460/) (2022)
- [KPI Extraction from Maintenance Work Orders -- A Comparison of Expert Labeling, Text Classification and AI-Assisted Tagging for Computing Failure Rates of Wind Turbines](https://mlsanity.com/paper/2311.04064/) (2023)
- [Actively Discovering New Slots for Task-oriented Conversation](https://mlsanity.com/paper/2305.04049/) (2023)

## Abstract (arXiv)

While deep learning is a powerful tool for natural language processing (NLP) problems, successful solutions to these problems rely heavily on large amounts of annotated samples. However, manually annotating data is expensive and time-consuming. Active Learning (AL) strategies reduce the need for huge volumes of labeled data by iteratively selecting a small number of examples for manual annotation based on their estimated utility in training the given model. In this paper, we argue that since AL strategies choose examples independently, they may potentially select similar examples, all of which may not contribute significantly to the learning process. Our proposed approach, Active2 Learning (A2L), actively adapts to the deep learning model being trained to eliminate further such redundant examples chosen by an AL strategy. We show that A2L is widely applicable by using it in conjunction with several different AL strategies and NLP tasks. We empirically demonstrate that the proposed approach is further able to reduce the data requirements of state-of-the-art AL strategies by an absolute percentage reduction of ≈3-25% on multiple NLP tasks while achieving the same performance with no additional computation overhead.