# AFD-SLU: Adaptive Feature Distillation for Spoken Language Understanding

arXiv: https://arxiv.org/abs/2509.04821 · HTML: https://mlsanity.com/paper/2509.04821/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 14 of 14)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations](https://mlsanity.com/paper/2407.04069/) (2024)
- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling](https://mlsanity.com/paper/1812.10235/) (2018)
- [GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2106.01925/) (2021)
- [Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding](https://mlsanity.com/paper/2112.11953/) (2021)
- [ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent](https://mlsanity.com/paper/2403.04481/) (2024)
- [Zero-shot Slot Filling in the Age of LLMs for Dialogue Systems](https://mlsanity.com/paper/2411.18980/) (2024)
- [JPIS: A Joint Model for Profile-based Intent Detection and Slot Filling with Slot-to-Intent Attention](https://mlsanity.com/paper/2312.08737/) (2023)
- [Pro-HAN: A Heterogeneous Graph Attention Network for Profile-Based Spoken Language Understanding](https://mlsanity.com/paper/2402.03900/) (2024)
- [MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU](https://mlsanity.com/paper/2408.08144/) (2024)

## Built on by (most cited first, 1 of 1)

- [Speech2Grasp: Data-Efficient Transfer of Text-Conditioned Grasp Detection to Speech in Humanoid Robots](https://mlsanity.com/paper/2607.26567/) (2026)

## Abstract (arXiv)

Spoken Language Understanding (SLU) is a core component of conversational systems, enabling machines to interpret user utterances. Despite its importance, developing effective SLU systems remains challenging due to the scarcity of labeled training data and the computational burden of deploying Large Language Models (LLMs) in real-world applications. To further alleviate these issues, we propose an Adaptive Feature Distillation framework that transfers rich semantic representations from a General Text Embeddings (GTE)-based teacher model to a lightweight student model. Our method introduces a dynamic adapter equipped with a Residual Projection Neural Network (RPNN) to align heterogeneous feature spaces, and a Dynamic Distillation Coefficient (DDC) that adaptively modulates the distillation strength based on real-time feedback from intent and slot prediction performance. Experiments on the Chinese profile-based ProSLU benchmark demonstrate that AFD-SLU achieves state-of-the-art results, with 95.67% intent accuracy, 92.02% slot F1 score, and 85.50% overall accuracy.