# ALT: An Automatic System for Long Tail Scenario Modeling

arXiv: https://arxiv.org/abs/2305.11390 · HTML: https://mlsanity.com/paper/2305.11390/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 71 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [Adversarial Examples Improve Image Recognition](https://mlsanity.com/paper/1911.09665/) (2019)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)
- [MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation](https://mlsanity.com/paper/1908.00413/) (2019)
- [One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction](https://mlsanity.com/paper/2101.11427/) (2021)
- [Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddings](https://mlsanity.com/paper/1904.11547/) (2019)
- [Automated Machine Learning: From Principles to Practices](https://mlsanity.com/paper/1810.13306/) (2018)
- [AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search](https://mlsanity.com/paper/2001.04246/) (2020)
- [Semi-Supervised Neural Architecture Search](https://mlsanity.com/paper/2002.10389/) (2020)
- [Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks](https://mlsanity.com/paper/2105.04790/) (2021)
- [Hyperparameter Optimization: A Spectral Approach](https://mlsanity.com/paper/1706.00764/) (2017)
- [A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Apps](https://mlsanity.com/paper/2106.03819/) (2021)
- [SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks](https://mlsanity.com/paper/2003.02556/) (2020)
- [Attentive Knowledge-aware Graph Convolutional Networks with Collaborative Guidance for Personalized Recommendation](https://mlsanity.com/paper/2109.02046/) (2021)
- [MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data](https://mlsanity.com/paper/2110.14354/) (2021)
- [Distributed Deep Forest and its Application to Automatic Detection of Cash-out Fraud](https://mlsanity.com/paper/1805.04234/) (2018)
- [We Know What You Want: An Advertising Strategy Recommender System for Online Advertising](https://mlsanity.com/paper/2105.14188/) (2021)
- [How Much Can A Retailer Sell? Sales Forecasting on Tmall](https://mlsanity.com/paper/2002.11940/) (2020)

## Abstract (arXiv)

In this paper, we consider the problem of long tail scenario modeling with budget limitation, i.e., insufficient human resources for model training stage and limited time and computing resources for model inference stage. This problem is widely encountered in various applications, yet has received deficient attention so far. We present an automatic system named ALT to deal with this problem. Several efforts are taken to improve the algorithms used in our system, such as employing various automatic machine learning related techniques, adopting the meta learning philosophy, and proposing an essential budget-limited neural architecture search method, etc. Moreover, to build the system, many optimizations are performed from a systematic perspective, and essential modules are armed, making the system more feasible and efficient. We perform abundant experiments to validate the effectiveness of our system and demonstrate the usefulness of the critical modules in our system. Moreover, online results are provided, which fully verified the efficacy of our system.