# REMEDI: Relative Feature Enhanced Meta-Learning with Distillation for Imbalanced Prediction

arXiv: https://arxiv.org/abs/2505.07245 · HTML: https://mlsanity.com/paper/2505.07245/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 58 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 9 of 9)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Delving Deep into Label Smoothing](https://mlsanity.com/paper/2011.12562/) (2020)
- [Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models](https://mlsanity.com/paper/2306.10933/) (2023)
- [Large-scale Causal Approaches to Debiasing Post-click Conversion Rate Estimation with Multi-task Learning](https://mlsanity.com/paper/1910.09337/) (2019)
- [A Unified Framework for Multi-Domain CTR Prediction via Large Language Models](https://mlsanity.com/paper/2312.10743/) (2023)
- [Adapting Software Architectures to Machine Learning Challenges](https://mlsanity.com/paper/2105.12422/) (2021)
- [Relation-aware Heterogeneous Graph for User Profiling](https://mlsanity.com/paper/2110.07181/) (2021)

## Abstract (arXiv)

Predicting future vehicle purchases among existing owners presents a critical challenge due to extreme class imbalance (<0.5% positive rate) and complex behavioral patterns. We propose REMEDI (Relative feature Enhanced Meta-learning with Distillation for Imbalanced prediction), a novel multi-stage framework addressing these challenges. REMEDI first trains diverse base models to capture complementary aspects of user behavior. Second, inspired by comparative op-timization techniques, we introduce relative performance meta-features (deviation from ensemble mean, rank among peers) for effective model fusion through a hybrid-expert architecture. Third, we distill the ensemble's knowledge into a single efficient model via supervised fine-tuning with MSE loss, enabling practical deployment. Evaluated on approximately 800,000 vehicle owners, REMEDI significantly outperforms baseline approaches, achieving the business target of identifying ~50% of actual buyers within the top 60,000 recommendations at ~10% precision. The distilled model preserves the ensemble's predictive power while maintaining deployment efficiency, demonstrating REMEDI's effectiveness for imbalanced prediction in industry settings.