# AdaFPP: Adapt-Focused Bi-Propagating Prototype Learning for Panoramic Activity Recognition

arXiv: https://arxiv.org/abs/2405.02538 · HTML: https://mlsanity.com/paper/2405.02538/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 4 May 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles](https://mlsanity.com/paper/2306.00989/) (2023)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [Cascaded Partial Decoder for Fast and Accurate Salient Object Detection](https://mlsanity.com/paper/1904.08739/) (2019)
- [MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition](https://mlsanity.com/paper/2201.08383/) (2022)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [Actor-Transformers for Group Activity Recognition](https://mlsanity.com/paper/2003.12737/) (2020)
- [Clustered Object Detection in Aerial Images](https://mlsanity.com/paper/1904.08008/) (2019)
- [PYSKL: Towards Good Practices for Skeleton Action Recognition](https://mlsanity.com/paper/2205.09443/) (2022)
- [Density Map Guided Object Detection in Aerial Images](https://mlsanity.com/paper/2004.05520/) (2020)
- [JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection](https://mlsanity.com/paper/2106.08827/) (2021)
- [Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition](https://mlsanity.com/paper/1811.00270/) (2018)
- [UFPMP-Det: Toward Accurate and Efficient Object Detection on Drone Imagery](https://mlsanity.com/paper/2112.10415/) (2021)
- [STMixer: A One-Stage Sparse Action Detector](https://mlsanity.com/paper/2303.15879/) (2023)
- [Focus-and-Detect: A Small Object Detection Framework for Aerial Images](https://mlsanity.com/paper/2203.12976/) (2022)
- [Multi-Label Activity Recognition using Activity-specific Features and Activity Correlations](https://mlsanity.com/paper/2009.07420/) (2020)
- [Human Interaction Recognition Framework based on Interacting Body Part Attention](https://mlsanity.com/paper/2101.08967/) (2021)
- [AdaZoom: Adaptive Zoom Network for Multi-Scale Object Detection in Large Scenes](https://mlsanity.com/paper/2106.10409/) (2021)
- [COCO: Performance Assessment](https://mlsanity.com/paper/1605.03560/) (2016)
- [Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration](https://mlsanity.com/paper/2105.14683/) (2021)
- [A Simple and Efficient Pipeline to Build an End-to-End Spatial-Temporal Action Detector](https://mlsanity.com/paper/2206.03064/) (2022)

## Built on by (most cited first, 2 of 2)

- [Fine-grained Image Retrieval via Dual-Vision Adaptation](https://mlsanity.com/paper/2506.16273/) (2025)
- [Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling](https://mlsanity.com/paper/2606.27745/) (2026)

## Abstract (arXiv)

Panoramic Activity Recognition (PAR) aims to identify multi-granularity behaviors performed by multiple persons in panoramic scenes, including individual activities, group activities, and global activities. Previous methods 1) heavily rely on manually annotated detection boxes in training and inference, hindering further practical deployment; or 2) directly employ normal detectors to detect multiple persons with varying size and spatial occlusion in panoramic scenes, blocking the performance gain of PAR. To this end, we consider learning a detector adapting varying-size occluded persons, which is optimized along with the recognition module in the all-in-one framework. Therefore, we propose a novel Adapt-Focused bi-Propagating Prototype learning (AdaFPP) framework to jointly recognize individual, group, and global activities in panoramic activity scenes by learning an adapt-focused detector and multi-granularity prototypes as the pretext tasks in an end-to-end way. Specifically, to accommodate the varying sizes and spatial occlusion of multiple persons in crowed panoramic scenes, we introduce a panoramic adapt-focuser, achieving the size-adapting detection of individuals by comprehensively selecting and performing fine-grained detections on object-dense sub-regions identified through original detections. In addition, to mitigate information loss due to inaccurate individual localizations, we introduce a bi-propagation prototyper that promotes closed-loop interaction and informative consistency across different granularities by facilitating bidirectional information propagation among the individual, group, and global levels. Extensive experiments demonstrate the significant performance of AdaFPP and emphasize its powerful applicability for PAR.