# Bitstream Action Recognition is Byte Modeling

arXiv: https://arxiv.org/abs/2608.15695 · HTML: https://mlsanity.com/paper/2608.15695/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers](https://mlsanity.com/paper/2305.07185/) (2023)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [Deep Residual Learning in the JPEG Transform Domain](https://mlsanity.com/paper/1812.11690/) (2018)
- [Contextual Action Recognition with R*CNN](https://mlsanity.com/paper/1505.01197/) (2015)
- [Beyond Language Models: Byte Models are Digital World Simulators](https://mlsanity.com/paper/2402.19155/) (2024)
- [Bytes Are All You Need: Transformers Operating Directly On File Bytes](https://mlsanity.com/paper/2306.00238/) (2023)
- [Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method](https://mlsanity.com/paper/2309.13890/) (2023)
- [Bitstream-Corrupted JPEG Images are Restorable: Two-stage Compensation and Alignment Framework for Image Restoration](https://mlsanity.com/paper/2304.06976/) (2023)
- [Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling](https://mlsanity.com/paper/2502.14553/) (2025)
- [ByteNet: Rethinking Multimedia File Fragment Classification through Visual Perspectives](https://mlsanity.com/paper/2410.20855/) (2024)
- [Training-Free Zero-Shot Temporal Action Detection with Vision-Language Models](https://mlsanity.com/paper/2501.13795/) (2025)
- [PromptSR: Cascade Prompting for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2507.04118/) (2025)
- [Transform and Bitstream Domain Image Classification](https://mlsanity.com/paper/2110.06740/) (2021)

## Built on by (most cited first, 2 of 2)

- [Towards Bitstream-corrupted Harsh Visual Understanding: Through Bitstream Language Modeling as Robust Semantic Priors](https://mlsanity.com/paper/2608.21837/) (2026)
- [ByteAction: Byte-space Action Recognition Foundation Model](https://mlsanity.com/paper/2608.22760/) (2026)

## Abstract (arXiv)

Conventional action recognition typically relies on successful pixel decoding of the bitstream. However, bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure, posing a significant challenge to reliable action recognition. Bitstream Action Recognition (BAR) aims to overcome the dependency on decoding and the vulnerability to corruption. In this paper, we propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE). BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action. This guides the generation of rich and stable representations for robustness to corruption through Intact-Anchored Representation Alignment (IARA). The intact representation serves as a stable anchor, and the corrupted one is aligned to it at the embedding and decision levels under Unreliable-Anchor Suppression (UAS), entirely in representation space and without repairing the bitstream. To address the scarcity of corrupted bitstreams in practice, we introduce the Real-world Bitstream Corruption Simulator (RBCS), a four-parameter simulator that reproduces bit-flip and byte-loss errors arising in transmission and storage. Building on RBCS, we construct the first large-scale BAR dataset (BAR-D), which comprises the BAR-Stanford40 and BAR-PPMI subsets and spans diverse corruption types and severity levels. Finally, we build a large benchmark on BAR-D involving 14 action recognition methods from the pixel, compressed, and bitstream domains. Extensive experiments demonstrate that BRACE has superior robustness to bitstream corruption than all comparison methods. Ablation studies further validate the effectiveness of the proposed RBCS augmentation and IARA.