# QueryMamba: A Mamba-Based Encoder-Decoder Architecture with a Statistical Verb-Noun Interaction Module for Video Action Forecasting @ Ego4D Long-Term Action Anticipation Challenge 2024

arXiv: https://arxiv.org/abs/2407.04184 · HTML: https://mlsanity.com/paper/2407.04184/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 12 of 12)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [When will you do what? - Anticipating Temporal Occurrences of Activities](https://mlsanity.com/paper/1804.00892/) (2018)
- [Future Transformer for Long-term Action Anticipation](https://mlsanity.com/paper/2205.14022/) (2022)
- [InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges](https://mlsanity.com/paper/2211.09529/) (2022)
- [Memory-and-Anticipation Transformer for Online Action Understanding](https://mlsanity.com/paper/2308.07893/) (2023)
- [DiffAnt: Diffusion Models for Action Anticipation](https://mlsanity.com/paper/2311.15991/) (2023)

## Built on by (most cited first, 4 of 4)

- [QMambaBSR: Burst Image Super-Resolution with Query State Space Model](https://mlsanity.com/paper/2408.08665/) (2024)
- [Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025](https://mlsanity.com/paper/2506.02550/) (2025)
- [Vision and Intention Boost Large Language Model in Long-Term Action Anticipation](https://mlsanity.com/paper/2505.01713/) (2025)
- [Reliable Egocentric Action Anticipation via Temporal Reliability Suppression and Compositional Graph Decoding](https://mlsanity.com/paper/2609.13293/) (2026)

## Abstract (arXiv)

This report presents a novel Mamba-based encoder-decoder architecture, QueryMamba, featuring an integrated verb-noun interaction module that utilizes a statistical verb-noun co-occurrence matrix to enhance video action forecasting. This architecture not only predicts verbs and nouns likely to occur based on historical data but also considers their joint occurrence to improve forecast accuracy. The efficacy of this approach is substantiated by experimental results, with the method achieving second place in the Ego4D LTA challenge and ranking first in noun prediction accuracy.