# DUET -- Dual User Embedding Transformers for Offsite Conversion Prediction

arXiv: https://arxiv.org/abs/2606.10243 · HTML: https://mlsanity.com/paper/2606.10243/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 30 of 31)

- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://mlsanity.com/paper/1804.07931/) (2018)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction](https://mlsanity.com/paper/2101.11427/) (2021)
- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://mlsanity.com/paper/2302.01115/) (2023)
- [ESCM2: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation](https://mlsanity.com/paper/2204.05125/) (2022)
- [LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders](https://mlsanity.com/paper/2505.04421/) (2025)
- [Contrastive Cross-domain Recommendation in Matching](https://mlsanity.com/paper/2112.00999/) (2021)
- [Scaling Law of Large Sequential Recommendation Models](https://mlsanity.com/paper/2311.11351/) (2023)
- [Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta](https://mlsanity.com/paper/2311.09544/) (2023)
- [TDR-CL: Targeted Doubly Robust Collaborative Learning for Debiased Recommendations](https://mlsanity.com/paper/2203.10258/) (2022)
- [A Generalized Doubly Robust Learning Framework for Debiasing Post-Click Conversion Rate Prediction](https://mlsanity.com/paper/2211.06684/) (2022)
- [A Pre-trained Sequential Recommendation Framework: Popularity Dynamics for Zero-shot Transfer](https://mlsanity.com/paper/2401.01497/) (2024)
- [Feature Decomposition for Reducing Negative Transfer: A Novel Multi-task Learning Method for Recommender System](https://mlsanity.com/paper/2302.05031/) (2023)
- [Async Learned User Embeddings for Ads Delivery Optimization](https://mlsanity.com/paper/2406.05898/) (2024)
- [LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation](https://mlsanity.com/paper/2601.20083/) (2026)
- [Masked Multi-Domain Network: Multi-Type and Multi-Scenario Conversion Rate Prediction with a Single Model](https://mlsanity.com/paper/2403.17425/) (2024)
- [Entire-Space Variational Information Exploitation for Post-Click Conversion Rate Prediction](https://mlsanity.com/paper/2502.15687/) (2024)
- [SIDE: Semantic ID Embedding for effective learning from sequences](https://mlsanity.com/paper/2506.16698/) (2025)
- [Contrastive Learning for Conversion Rate Prediction](https://mlsanity.com/paper/2307.05974/) (2023)
- [Improving conversion rate prediction via self-supervised pre-training in online advertising](https://mlsanity.com/paper/2401.16432/) (2024)
- [Adaptive Domain Scaling for Personalized Sequential Modeling in Recommenders](https://mlsanity.com/paper/2502.05523/) (2025)
- [Enhancing Transferability and Consistency in Cross-Domain Recommendations via Supervised Disentanglement](https://mlsanity.com/paper/2507.17112/) (2025)
- [Multi-Faceted Large Embedding Tables for Pinterest Ads Ranking](https://mlsanity.com/paper/2508.05700/) (2025)

## Abstract (arXiv)

Offsite conversion rate (OCVR) prediction is an important ranking problem in computational recommendation systems. This task presents a modeling challenge: click signals are abundant and exhibit short temporal horizons, whereas conversion signals are inherently sparse, long-delayed, and frequently unattributed. Despite these statistical disparities, both signal types must inform models that operate within strict serving-latency constraints. Prior pre-training approaches address this heterogeneity with a single, undifferentiated encoder applied uniformly across both data streams. We propose DUET (Dual User Embedding Transformers), a framework that explicitly partitions user behavioral data into two domain-coherent streams -- clicks and conversions -- and pre-trains dedicated transformer encoders with architectures tailored to each stream's statistical characteristics: multi-layer self-attention for the dense click stream and interleaved cross- and self-attention for the sparse conversion stream. The resulting complementary embeddings are jointly consumed by a downstream ranker without exceeding serving-latency budgets. Evaluation demonstrates up to 0.38% normalized entropy (NE) reduction relative to the strongest baseline, and A/B test shows consistent improvements in OCVR prediction accuracy.