# Deep Task-specific Bottom Representation Network for Multi-Task Recommendation

arXiv: https://arxiv.org/abs/2308.05996 · HTML: https://mlsanity.com/paper/2308.05996/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 11 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://mlsanity.com/paper/1804.07931/) (2018)
- [Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models](https://mlsanity.com/paper/2010.05874/) (2020)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09433/) (2019)
- [Deep Session Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.06482/) (2019)
- [ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation](https://mlsanity.com/paper/1711.06632/) (2017)
- [User Behavior Retrieval for Click-Through Rate Prediction](https://mlsanity.com/paper/2005.14171/) (2020)
- [Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://mlsanity.com/paper/2105.08489/) (2021)
- [Hierarchically Modeling Micro and Macro Behaviors via Multi-Task Learning for Conversion Rate Prediction](https://mlsanity.com/paper/2104.09713/) (2021)
- [Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling](https://mlsanity.com/paper/2201.06814/) (2022)
- [PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems](https://mlsanity.com/paper/2204.04903/) (2022)
- [Learning to Recommend with Multiple Cascading Behaviors](https://mlsanity.com/paper/1809.08161/) (2018)
- [Automatic Expert Selection for Multi-Scenario and Multi-Task Search](https://mlsanity.com/paper/2205.14321/) (2022)
- [Fast Variational AutoEncoder with Inverted Multi-Index for Collaborative Filtering](https://mlsanity.com/paper/2109.05773/) (2021)
- [A Multi-task Learning Framework for Product Ranking with BERT](https://mlsanity.com/paper/2202.05317/) (2022)
- [Boosting Share Routing for Multi-task Learning](https://mlsanity.com/paper/2009.00387/) (2020)

## Built on by (most cited first, 5 of 5)

- [HoME: Hierarchy of Multi-Gate Experts for Multi-Task Learning at Kuaishou](https://mlsanity.com/paper/2408.05430/) (2024)
- [A Survey of Controllable Learning: Methods and Applications in Information Retrieval](https://mlsanity.com/paper/2407.06083/) (2024)
- [Mixed Supervised Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2404.15954/) (2024)
- [TayFCS: Towards Light Feature Combination Selection for Deep Recommender Systems](https://mlsanity.com/paper/2507.03895/) (2025)
- [Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning](https://mlsanity.com/paper/2607.14192/) (2026)

## Abstract (arXiv)

Neural-based multi-task learning (MTL) has gained significant improvement, and it has been successfully applied to recommendation system (RS). Recent deep MTL methods for RS (e.g. MMoE, PLE) focus on designing soft gating-based parameter-sharing networks that implicitly learn a generalized representation for each task. However, MTL methods may suffer from performance degeneration when dealing with conflicting tasks, as negative transfer effects can occur on the task-shared bottom representation. This can result in a reduced capacity for MTL methods to capture task-specific characteristics, ultimately impeding their effectiveness and hindering the ability to generalize well on all tasks. In this paper, we focus on the bottom representation learning of MTL in RS and propose the Deep Task-specific Bottom Representation Network (DTRN) to alleviate the negative transfer problem. DTRN obtains task-specific bottom representation explicitly by making each task have its own representation learning network in the bottom representation modeling stage. Specifically, it extracts the user's interests from multiple types of behavior sequences for each task through the parameter-efficient hypernetwork. To further obtain the dedicated representation for each task, DTRN refines the representation of each feature by employing a SENet-like network for each task. The two proposed modules can achieve the purpose of getting task-specific bottom representation to relieve tasks' mutual interference. Moreover, the proposed DTRN is flexible to combine with existing MTL methods. Experiments on one public dataset and one industrial dataset demonstrate the effectiveness of the proposed DTRN.