# Streaming Anchor Loss: Augmenting Supervision with Temporal Significance

arXiv: https://arxiv.org/abs/2310.05886 · HTML: https://mlsanity.com/paper/2310.05886/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 13 of 13)

- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018)
- [Alleviating Over-segmentation Errors by Detecting Action Boundaries](https://mlsanity.com/paper/2007.06866/) (2020)
- [InverseForm: A Loss Function for Structured Boundary-Aware Segmentation](https://mlsanity.com/paper/2104.02745/) (2021)
- [Multimodal Inductive Transfer Learning for Detection of Alzheimer's Dementia and its Severity](https://mlsanity.com/paper/2009.00700/) (2020)
- [Anchor Loss: Modulating Loss Scale based on Prediction Difficulty](https://mlsanity.com/paper/1909.11155/) (2019)
- [Towards duration robust weakly supervised sound event detection](https://mlsanity.com/paper/2101.07687/) (2021)
- [Optimize what matters: Training DNN-HMM Keyword Spotting Model Using End Metric](https://mlsanity.com/paper/2011.01151/) (2020)
- [Stacked 1D convolutional networks for end-to-end small footprint voice trigger detection](https://mlsanity.com/paper/2008.03405/) (2020)
- [Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation](https://mlsanity.com/paper/2105.06598/) (2021)
- [Dynamic curriculum learning via data parameters for noise robust keyword spotting](https://mlsanity.com/paper/2102.09666/) (2021)
- [Efficient Multimodal Neural Networks for Trigger-less Voice Assistants](https://mlsanity.com/paper/2305.12063/) (2023)
- [WakeUpNet: A Mobile-Transformer based Framework for End-to-End Streaming Voice Trigger](https://mlsanity.com/paper/2210.02904/) (2022)

## Built on by (most cited first, 1 of 1)

- [Comparative Analysis of Personalized Voice Activity Detection Systems: Assessing Real-World Effectiveness](https://mlsanity.com/paper/2406.09443/) (2024)

## Abstract (arXiv)

Streaming neural network models for fast frame-wise responses to various speech and sensory signals are widely adopted on resource-constrained platforms. Hence, increasing the learning capacity of such streaming models (i.e., by adding more parameters) to improve the predictive power may not be viable for real-world tasks. In this work, we propose a new loss, Streaming Anchor Loss (SAL), to better utilize the given learning capacity by encouraging the model to learn more from essential frames. More specifically, our SAL and its focal variations dynamically modulate the frame-wise cross entropy loss based on the importance of the corresponding frames so that a higher loss penalty is assigned for frames within the temporal proximity of semantically critical events. Therefore, our loss ensures that the model training focuses on predicting the relatively rare but task-relevant frames. Experimental results with standard lightweight convolutional and recurrent streaming networks on three different speech based detection tasks demonstrate that SAL enables the model to learn the overall task more effectively with improved accuracy and latency, without any additional data, model parameters, or architectural changes.