# JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling

arXiv: https://arxiv.org/abs/2606.19108 · HTML: https://mlsanity.com/paper/2606.19108/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Intent Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2202.02519/) (2022)
- [Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1706.04148/) (2017)
- [Translation-based Recommendation](https://mlsanity.com/paper/1707.02410/) (2017)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)
- [Frequency Enhanced Hybrid Attention Network for Sequential Recommendation](https://mlsanity.com/paper/2304.09184/) (2023)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://mlsanity.com/paper/2306.00248/) (2023)
- [Applying Deep Learning To Airbnb Search](https://mlsanity.com/paper/1810.09591/) (2018)
- [Attention Calibration for Transformer-based Sequential Recommendation](https://mlsanity.com/paper/2308.09419/) (2023)
- [Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems](https://mlsanity.com/paper/2305.12102/) (2023)
- [Managing Diversity in Airbnb Search](https://mlsanity.com/paper/2004.02621/) (2020)
- [RankFormer: Listwise Learning-to-Rank Using Listwide Labels](https://mlsanity.com/paper/2306.05808/) (2023)
- [Optimizing Airbnb Search Journey with Multi-task Learning](https://mlsanity.com/paper/2305.18431/) (2023)
- [Multi-objective Learning to Rank by Model Distillation](https://mlsanity.com/paper/2407.07181/) (2024)
- [Learning To Rank Diversely At Airbnb](https://mlsanity.com/paper/2210.07774/) (2022)
- [Semantic Retrieval at Walmart](https://mlsanity.com/paper/2412.04637/) (2024)

## Abstract (arXiv)

Sequence modeling has become increasingly popular in recommendation and ranking algorithms, owing to its capacity to model users' historical behaviors and infer user intentions. Despite its theoretical simplicity, the practical deployment of a sequence model in production is non-trivial due to complexity of the sequence and sparse labels. For example, in Airbnb, guest sequences are often long, exploratory and complex, and we focus on booking labels, which are sparse. As such, we are often required to make various design decisions regarding data and modeling to strike a balance between effectiveness and scalability. This work delved into these production challenges and deployed JourneyFormer, a sequence modeling solution for search ranking at Airbnb. We detail crucial design considerations, covering aspects such as guest event selection, ID embeddings, model architecture, and label attribution. Additionally, we describe several tailored strategies to accelerate model training and inference. JourneyFormer has been successfully deployed within Airbnb's production, where its effectiveness and impact have been evidenced not only by improved offline ranking metrics but also by significant gains in key business metrics through online A/B testing across 2 production surfaces.