# KMM: Key Frame Mask Mamba for Extended Motion Generation

arXiv: https://arxiv.org/abs/2411.06481 · HTML: https://mlsanity.com/paper/2411.06481/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 41 of its 49 references that are in the map.
- First posted on arXiv 10 Nov 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Video, motion and generative Mamba models](https://mlsanity.com/ssm/video-motion-and-generative-mamba-models/)

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [TEACH: Temporal Action Composition for 3D Humans](https://mlsanity.com/paper/2209.04066/) (2022)
- [AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism](https://mlsanity.com/paper/2309.00796/) (2023)
- [Motion Mamba: Efficient and Long Sequence Motion Generation](https://mlsanity.com/paper/2403.07487/) (2024)
- [DiffCollage: Parallel Generation of Large Content with Diffusion Models](https://mlsanity.com/paper/2303.17076/) (2023)
- [Unified Human-Scene Interaction via Prompted Chain-of-Contacts](https://mlsanity.com/paper/2309.07918/) (2023)
- [Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation](https://mlsanity.com/paper/2401.08559/) (2024)
- [FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba](https://mlsanity.com/paper/2404.09498/) (2024)
- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration](https://mlsanity.com/paper/2304.02419/) (2023)

## Built on by (most cited first, 16 of 16)

- [Motion Anything: Any to Motion Generation](https://mlsanity.com/paper/2503.06955/) (2025)
- [PresentAgent: Multimodal Agent for Presentation Video Generation](https://mlsanity.com/paper/2507.04036/) (2025)
- [ReMoMask: Retrieval-Augmented Masked Motion Generation](https://mlsanity.com/paper/2508.02605/) (2025)
- [Motion-R1: Enhancing Motion Generation with Decomposed Chain-of-Thought and RL Binding](https://mlsanity.com/paper/2506.10353/) (2025)
- [SafeMo: Linguistically Grounded Unlearning for Trustworthy Text-to-Motion Generation](https://mlsanity.com/paper/2601.00590/) (2026)
- [PresentAgent-2: Towards Generalist Multimodal Presentation Agents](https://mlsanity.com/paper/2605.11363/) (2026)
- [InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mamba](https://mlsanity.com/paper/2506.03084/) (2025)
- [Language-Guided Transformer Tokenizer for Human Motion Generation](https://mlsanity.com/paper/2602.08337/) (2026)
- [Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors](https://mlsanity.com/paper/2603.29655/) (2026)
- [UniMesh: Unifying 3D Mesh Understanding and Generation](https://mlsanity.com/paper/2604.17472/) (2026)
- [A Novel Convolutional-Free Method for 3D Medical Imaging Segmentation](https://mlsanity.com/paper/2502.05396/) (2025)
- [Human Motion Prediction, Reconstruction, and Generation](https://mlsanity.com/paper/2502.15956/) (2025)
- [PartRAG: Retrieval-Augmented Part-Level 3D Generation and Editing](https://mlsanity.com/paper/2602.17033/) (2026)
- [T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation](https://mlsanity.com/paper/2602.01352/) (2026)
- [MoRL: Reinforced Reasoning for Unified Motion Understanding and Generation](https://mlsanity.com/paper/2602.14534/) (2026)
- [UniMo: Unifying Human and Animal Motion Generation](https://mlsanity.com/paper/2609.12342/) (2026)

## Abstract (arXiv)

Human motion generation is a cut-edge area of research in generative computer vision, with promising applications in video creation, game development, and robotic manipulation. The recent Mamba architecture shows promising results in efficiently modeling long and complex sequences, yet two significant challenges remain: Firstly, directly applying Mamba to extended motion generation is ineffective, as the limited capacity of the implicit memory leads to memory decay. Secondly, Mamba struggles with multimodal fusion compared to Transformers, and lack alignment with textual queries, often confusing directions (left or right) or omitting parts of longer text queries. To address these challenges, our paper presents three key contributions: Firstly, we introduce KMM, a novel architecture featuring Key frame Masking Modeling, designed to enhance Mamba's focus on key actions in motion segments. This approach addresses the memory decay problem and represents a pioneering method in customizing strategic frame-level masking in SSMs. Additionally, we designed a contrastive learning paradigm for addressing the multimodal fusion problem in Mamba and improving the motion-text alignment. Finally, we conducted extensive experiments on the go-to dataset, BABEL, achieving state-of-the-art performance with a reduction of more than 57% in FID and 70% parameters compared to previous state-of-the-art methods. See project website: https://steve-zeyu-zhang.github.io/KMM