# Dance with Flow: Two-in-One Stream Action Detection

arXiv: https://arxiv.org/abs/1904.00696 · HTML: https://mlsanity.com/paper/1904.00696/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2019.

## Built on by (most cited first, 30 of 30)

- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [TubeR: Tubelet Transformer for Video Action Detection](https://mlsanity.com/paper/2104.00969/) (2021)
- [ROAD: The ROad event Awareness Dataset for Autonomous Driving](https://mlsanity.com/paper/2102.11585/) (2021)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition](https://mlsanity.com/paper/2110.10101/) (2021)
- [Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection](https://mlsanity.com/paper/2108.03619/) (2021)
- [End-to-End Semi-Supervised Learning for Video Action Detection](https://mlsanity.com/paper/2203.04251/) (2022)
- [SLIC: Self-Supervised Learning with Iterative Clustering for Human Action Videos](https://mlsanity.com/paper/2206.12534/) (2022)
- [Object Priors for Classifying and Localizing Unseen Actions](https://mlsanity.com/paper/2104.04715/) (2021)
- [Spatio-Temporal Action Detection Under Large Motion](https://mlsanity.com/paper/2209.02250/) (2022)
- [Cross-Domain First Person Audio-Visual Action Recognition through Relative Norm Alignment](https://mlsanity.com/paper/2106.01689/) (2021)
- [Unsupervised 3D Pose Estimation for Hierarchical Dance Video Recognition](https://mlsanity.com/paper/2109.09166/) (2021)
- [Modality Mixer for Multi-modal Action Recognition](https://mlsanity.com/paper/2208.11314/) (2022)
- [Smart Director: An Event-Driven Directing System for Live Broadcasting](https://mlsanity.com/paper/2201.04024/) (2022)
- [ACDnet: An action detection network for real-time edge computing based on flow-guided feature approximation and memory aggregation](https://mlsanity.com/paper/2102.13493/) (2021)
- [Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting](https://mlsanity.com/paper/2106.10137/) (2021)
- [MINOTAUR: Multi-task Video Grounding From Multimodal Queries](https://mlsanity.com/paper/2302.08063/) (2023)
- [Egocentric zone-aware action recognition across environments](https://mlsanity.com/paper/2409.14205/) (2024)
- [A Survey on Deep Learning-based Spatio-temporal Action Detection](https://mlsanity.com/paper/2308.01618/) (2023)
- [EgoMusic-driven Human Dance Motion Estimation with Skeleton Mamba](https://mlsanity.com/paper/2508.10522/) (2025)
- [Point3D: tracking actions as moving points with 3D CNNs](https://mlsanity.com/paper/2203.10584/) (2022)
- [Feature-Supervised Action Modality Transfer](https://mlsanity.com/paper/2108.03329/) (2021)
- [GTM: Gray Temporal Model for Video Recognition](https://mlsanity.com/paper/2110.10348/) (2021)
- [RADNet: A Deep Neural Network Model for Robust Perception in Moving Autonomous Systems](https://mlsanity.com/paper/2205.00364/) (2022)
- [Modality Mixer Exploiting Complementary Information for Multi-modal Action Recognition](https://mlsanity.com/paper/2311.12344/) (2023)
- [STMixer: A One-Stage Sparse Action Detector](https://mlsanity.com/paper/2404.09842/) (2024)
- [JoVALE: Detecting Human Actions in Video Using Audiovisual and Language Contexts](https://mlsanity.com/paper/2412.13708/) (2024)
- [Dual Guidance Semi-Supervised Action Detection](https://mlsanity.com/paper/2507.21247/) (2025)
- [TubeLite: Lightweight Multi-Actor Spatio-Temporal Action Detection](https://mlsanity.com/paper/2607.04684/) (2026)
- [Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining](https://mlsanity.com/paper/2608.25701/) (2026)

## Abstract (arXiv)

The goal of this paper is to detect the spatio-temporal extent of an action. The two-stream detection network based on RGB and flow provides state-of-the-art accuracy at the expense of a large model-size and heavy computation. We propose to embed RGB and optical-flow into a single two-in-one stream network with new layers. A motion condition layer extracts motion information from flow images, which is leveraged by the motion modulation layer to generate transformation parameters for modulating the low-level RGB features. The method is easily embedded in existing appearance- or two-stream action detection networks, and trained end-to-end. Experiments demonstrate that leveraging the motion condition to modulate RGB features improves detection accuracy. With only half the computation and parameters of the state-of-the-art two-stream methods, our two-in-one stream still achieves impressive results on UCF101-24, UCFSports and J-HMDB.