# HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems

arXiv: https://arxiv.org/abs/2511.20235 · HTML: https://mlsanity.com/paper/2511.20235/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2025.

## Built on by (most cited first, 4 of 4)

- [UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems](https://mlsanity.com/paper/2604.00590/) (2026)
- [KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao](https://mlsanity.com/paper/2603.22779/) (2026)
- [SlimPer: Make Personalization Model Slim and Smart](https://mlsanity.com/paper/2607.12281/) (2026)
- [UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking](https://mlsanity.com/paper/2603.24226/) (2026)

## Abstract (arXiv)

We propose HHFT (Hierarchical Heterogeneous Feature Transformer), a Transformer-based architecture tailored for industrial CTR prediction. HHFT addresses the limitations of DNN through three key designs: (1) Semantic Feature Partitioning: Grouping heterogeneous features (e.g. user profile, item information, behaviour sequennce) into semantically coherent blocks to preserve domain-specific information; (2) Heterogeneous Transformer Encoder: Adopting block-specific QKV projections and FFNs to avoid semantic confusion between distinct feature types; (3) Hiformer Layer: Capturing high-order interactions across features. Our findings reveal that Transformers significantly outperform DNN baselines, achieving a +0.4% improvement in CTR AUC at scale. We have successfully deployed the model on Taobao's production platform, observing a significant uplift in key business metrics, including a +0.6% increase in Gross Merchandise Value (GMV).