# Exploring Ordinal Bias in Action Recognition for Instructional Videos

arXiv: https://arxiv.org/abs/2504.06580 · HTML: https://mlsanity.com/paper/2504.06580/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 9 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [Alleviating Over-segmentation Errors by Detecting Action Boundaries](https://mlsanity.com/paper/2007.06866/) (2020)
- [Diffusion Action Segmentation](https://mlsanity.com/paper/2303.17959/) (2023)
- [Uncovering Hidden Challenges in Query-Based Video Moment Retrieval](https://mlsanity.com/paper/2009.00325/) (2020)
- [Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos](https://mlsanity.com/paper/2203.14104/) (2022)
- [A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metric](https://mlsanity.com/paper/2101.09028/) (2021)
- [On the Consistency of Video Large Language Models in Temporal Comprehension](https://mlsanity.com/paper/2411.12951/) (2024)
- [SOAR: Scene-debiasing Open-set Action Recognition](https://mlsanity.com/paper/2309.01265/) (2023)
- [Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks](https://mlsanity.com/paper/2209.09393/) (2022)

## Built on by (most cited first, 1 of 1)

- [Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models](https://mlsanity.com/paper/2602.11244/) (2026)

## Abstract (arXiv)

Action recognition models have achieved promising results in understanding instructional videos. However, they often rely on dominant, dataset-specific action sequences rather than true video comprehension, a problem that we define as ordinal bias. To address this issue, we propose two effective video manipulation methods: Action Masking, which masks frames of frequently co-occurring actions, and Sequence Shuffling, which randomizes the order of action segments. Through comprehensive experiments, we demonstrate that current models exhibit significant performance drops when confronted with nonstandard action sequences, underscoring their vulnerability to ordinal bias. Our findings emphasize the importance of rethinking evaluation strategies and developing models capable of generalizing beyond fixed action patterns in diverse instructional videos.