# STEER: Semantic Turn Extension-Expansion Recognition for Voice Assistants

arXiv: https://arxiv.org/abs/2310.16990 · HTML: https://mlsanity.com/paper/2310.16990/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Semantic Parsing for Task Oriented Dialog using Hierarchical Representations](https://mlsanity.com/paper/1810.07942/) (2018)
- [Conversational Semantic Parsing for Dialog State Tracking](https://mlsanity.com/paper/2010.12770/) (2020)
- [Tree-structured Attention with Hierarchical Accumulation](https://mlsanity.com/paper/2002.08046/) (2020)
- [Turn-Taking Prediction for Natural Conversational Speech](https://mlsanity.com/paper/2208.13321/) (2022)
- [Dynamic Speech Endpoint Detection with Regression Targets](https://mlsanity.com/paper/2210.14252/) (2022)
- [Intelligent Assistant Language Understanding On Device](https://mlsanity.com/paper/2308.03905/) (2023)
- [Precise Detection of Speech Endpoints Dynamically: A Wavelet Convolution based approach](https://mlsanity.com/paper/1804.06159/) (2018)
- [Using Pause Information for More Accurate Entity Recognition](https://mlsanity.com/paper/2109.13222/) (2021)

## Built on by (most cited first, 1 of 1)

- [How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue](https://mlsanity.com/paper/2605.10199/) (2026)

## Abstract (arXiv)

In the context of a voice assistant system, steering refers to the phenomenon in which a user issues a follow-up command attempting to direct or clarify a previous turn. We propose STEER, a steering detection model that predicts whether a follow-up turn is a user's attempt to steer the previous command. Constructing a training dataset for steering use cases poses challenges due to the cold-start problem. To overcome this, we developed heuristic rules to sample opt-in usage data, approximating positive and negative samples without any annotation. Our experimental results show promising performance in identifying steering intent, with over 95% accuracy on our sampled data. Moreover, STEER, in conjunction with our sampling strategy, aligns effectively with real-world steering scenarios, as evidenced by its strong zero-shot performance on a human-graded evaluation set. In addition to relying solely on user transcripts as input, we introduce STEER+, an enhanced version of the model. STEER+ utilizes a semantic parse tree to provide more context on out-of-vocabulary words, such as named entities that often occur at the sentence boundary. This further improves model performance, reducing error rate in domains where entities frequently appear, such as messaging. Lastly, we present a data analysis that highlights the improvement in user experience when voice assistants support steering use cases.