# Born for Auto-Tagging: Faster and better with new objective functions

arXiv: https://arxiv.org/abs/2206.07264 · HTML: https://mlsanity.com/paper/2206.07264/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Understanding the Difficulty of Training Transformers](https://mlsanity.com/paper/2004.08249/) (2020)
- [The Evolved Transformer](https://mlsanity.com/paper/1901.11117/) (2019)
- [A Deep Relevance Matching Model for Ad-hoc Retrieval](https://mlsanity.com/paper/1711.08611/) (2017)
- [End-to-End Neural Ad-hoc Ranking with Kernel Pooling](https://mlsanity.com/paper/1706.06613/) (2017)
- [DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval](https://mlsanity.com/paper/1710.05649/) (2017)
- [MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning](https://mlsanity.com/paper/1911.09483/) (2019)
- [Progress Notes Classification and Keyword Extraction using Attention-based Deep Learning Models with BERT](https://mlsanity.com/paper/1910.05786/) (2019)

## Built on by (most cited first, 1 of 1)

- [Retrieval-Augmented Language Model for Extreme Multi-Label Knowledge Graph Link Prediction](https://mlsanity.com/paper/2405.12656/) (2024)

## Abstract (arXiv)

Keyword extraction is a task of text mining. It is applied to increase search volume in SEO and ads. Implemented in auto-tagging, it makes tagging on a mass scale of online articles and photos efficiently and accurately. BAT is invented for auto-tagging which served as awoo's AI marketing platform (AMP). awoo AMP not only provides service as a customized recommender system but also increases the converting rate in E-commerce. The strength of BAT converges faster and better than other SOTA models, as its 4-layer structure achieves the best F scores at 50 epochs. In other words, it performs better than other models which require deeper layers at 100 epochs. To generate rich and clean tags, awoo creates new objective functions to maintain similar rm F1 scores with cross-entropy while enhancing rm F2 scores simultaneously. To assure the even better performance of F scores awoo revamps the learning rate strategy proposed by Transformer citeTransformer to increase rm F1 and rm F2 scores at the same time.