# Sequential choice in ordered bundles

arXiv: https://arxiv.org/abs/2410.21670 · HTML: https://mlsanity.com/paper/2410.21670/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 72 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning](https://mlsanity.com/paper/2309.02591/) (2023)
- [VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2406.05370/) (2024)
- [A Survey on Session-based Recommender Systems](https://mlsanity.com/paper/1902.04864/) (2019)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective](https://mlsanity.com/paper/2312.15524/) (2023)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [Gated Channel Transformation for Visual Recognition](https://mlsanity.com/paper/1909.11519/) (2019)
- [The Music Streaming Sessions Dataset](https://mlsanity.com/paper/1901.09851/) (2018)
- [Why People Skip Music? On Predicting Music Skips using Deep Reinforcement Learning](https://mlsanity.com/paper/2301.03881/) (2023)
- [Session-based Sequential Skip Prediction via Recurrent Neural Networks](https://mlsanity.com/paper/1902.04743/) (2019)

## Abstract (arXiv)

Experience goods such as sporting and artistic events, songs, videos, news stories, podcasts, and television series, are often packaged and consumed in bundles. Many such bundles are ordered in the sense that the individual items are consumed sequentially, one at a time. We examine if an individual's decision to consume the next item in an ordered bundle can be predicted based on his/her consumption pattern for the preceding items. We evaluate several predictive models, including two custom Transformers using decoder-only and encoder-decoder architectures, fine-tuned GPT-3, a custom LSTM model, a reinforcement learning model, two Markov models, and a zero-order model. Using data from Spotify, we find that the custom Transformer with a decoder-only architecture provides the most accurate predictions, both for individual choices and aggregate demand. This model captures a general form of state dependence. Analysis of Transformer attention weights suggests that the consumption of the next item in a bundle is based on approximately equal weighting of all preceding choices. Our results indicate that the Transformer can assist in queuing the next item that an individual is likely to consume from an ordered bundle, predicting the demand for individual items, and personalizing promotions to increase demand.