# Temporal-Frequency State Space Duality: An Efficient Paradigm for Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2412.16904 · HTML: https://mlsanity.com/paper/2412.16904/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- First posted on arXiv 22 Dec 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Built on by (most cited first, 10 of 10)

- [MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion](https://mlsanity.com/paper/2507.08344/) (2025)
- [Online Micro-gesture Recognition Using Data Augmentation and Spatial-Temporal Attention](https://mlsanity.com/paper/2507.09512/) (2025)
- [MA-Bench: Towards Fine-grained Micro-Action Understanding](https://mlsanity.com/paper/2603.26586/) (2026)
- [OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs](https://mlsanity.com/paper/2503.21480/) (2025)
- [Task-Generalized Adaptive Cross-Domain Learning for Multimodal Image Fusion](https://mlsanity.com/paper/2508.15505/) (2025)
- [Multimodal Protein Language Models for Enzyme Kinetic Parameters: From Substrate Recognition to Conformational Adaptation](https://mlsanity.com/paper/2603.12845/) (2026)
- [AMMSM: Adaptive Motion Magnification and Sparse Mamba for Micro-Expression Recognition](https://mlsanity.com/paper/2503.24057/) (2025)
- [On the Contribution of Lexical Features to Speech Emotion Recognition](https://mlsanity.com/paper/2509.05634/) (2025)
- [Training-Free Multimodal Deepfake Detection via Graph Reasoning](https://mlsanity.com/paper/2509.21774/) (2025)
- [Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition](https://mlsanity.com/paper/2609.05755/) (2026)

## Abstract (arXiv)

Speech Emotion Recognition (SER) plays a critical role in enhancing user experience within human-computer interaction. However, existing methods are overwhelmed by temporal domain analysis, overlooking the valuable envelope structures of the frequency domain that are equally important for robust emotion recognition. To overcome this limitation, we propose TF-Mamba, a novel multi-domain framework that captures emotional expressions in both temporal and frequency dimensions.Concretely, we propose a temporal-frequency mamba block to extract temporal- and frequency-aware emotional features, achieving an optimal balance between computational efficiency and model expressiveness. Besides, we design a Complex Metric-Distance Triplet (CMDT) loss to enable the model to capture representative emotional clues for SER. Extensive experiments on the IEMOCAP and MELD datasets show that TF-Mamba surpasses existing methods in terms of model size and latency, providing a more practical solution for future SER applications.