# FTMoMamba: Motion Generation with Frequency and Text State Space Models

arXiv: https://arxiv.org/abs/2411.17532 · HTML: https://mlsanity.com/paper/2411.17532/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 9 of its 37 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 9 of 9)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation](https://mlsanity.com/paper/2406.04339/) (2024)
- [Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2404.04256/) (2024)
- [Fusion-Mamba for Cross-modality Object Detection](https://mlsanity.com/paper/2404.09146/) (2024)
- [FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining](https://mlsanity.com/paper/2405.19450/) (2024)
- [IRSRMamba: Infrared Image Super-Resolution via Mamba-based Wavelet Transform Feature Modulation Model](https://mlsanity.com/paper/2405.09873/) (2024)
- [Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition](https://mlsanity.com/paper/2209.09194/) (2022)
- [Alignment-free Raw Video Demoireing](https://mlsanity.com/paper/2408.10679/) (2024)

## Built on by (most cited first, 3 of 3)

- [Align Your Rhythm: Generating Highly Aligned Dance Poses with Gating-Enhanced Rhythm-Aware Feature Representation](https://mlsanity.com/paper/2503.17340/) (2025)
- [Hierarchical Relation-augmented Representation Generalization for Few-shot Action Recognition](https://mlsanity.com/paper/2504.10079/) (2025)
- [TriC-Motion: Tri-Domain Causal Modeling Grounded Text-to-Motion Generation](https://mlsanity.com/paper/2602.08462/) (2026)

## Abstract (arXiv)

Diffusion models achieve impressive performance in human motion generation. However, current approaches typically ignore the significance of frequency-domain information in capturing fine-grained motions within the latent space (e.g., low frequencies correlate with static poses, and high frequencies align with fine-grained motions). Additionally, there is a semantic discrepancy between text and motion, leading to inconsistency between the generated motions and the text descriptions. In this work, we propose a novel diffusion-based FTMoMamba framework equipped with a Frequency State Space Model (FreqSSM) and a Text State Space Model (TextSSM). Specifically, to learn fine-grained representation, FreqSSM decomposes sequences into low-frequency and high-frequency components, guiding the generation of static pose (e.g., sits, lay) and fine-grained motions (e.g., transition, stumble), respectively. To ensure the consistency between text and motion, TextSSM encodes text features at the sentence level, aligning textual semantics with sequential features. Extensive experiments show that FTMoMamba achieves superior performance on the text-to-motion generation task, especially gaining the lowest FID of 0.181 (rather lower than 0.421 of MLD) on the HumanML3D dataset.