# Multiview Detection with Shadow Transformer (and View-Coherent Data Augmentation)

arXiv: https://arxiv.org/abs/2108.05888 · HTML: https://mlsanity.com/paper/2108.05888/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Learning non-maximum suppression](https://mlsanity.com/paper/1705.02950/) (2017)
- [Multiview Detection with Feature Perspective Transformation](https://mlsanity.com/paper/2007.07247/) (2020)
- [Repulsion Loss: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1711.07752/) (2017)
- [Occlusion-aware R-CNN: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1807.08407/) (2018)
- [Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection](https://mlsanity.com/paper/1704.05775/) (2017)
- [Weakly Supervised 3D Object Detection from Point Clouds](https://mlsanity.com/paper/2007.13970/) (2020)
- [Deep Multi-camera People Detection](https://mlsanity.com/paper/1702.04593/) (2017)
- [Guided Attention Network for Object Detection and Counting on Drones](https://mlsanity.com/paper/1909.11307/) (2019)

## Built on by (most cited first, 30 of 45)

- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [Hierarchical Local-Global Transformer for Temporal Sentence Grounding](https://mlsanity.com/paper/2208.14882/) (2022)
- [EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View](https://mlsanity.com/paper/2310.13350/) (2023)
- [MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark](https://mlsanity.com/paper/2111.15157/) (2021)
- [Lifting Multi-View Detection and Tracking to the Bird's Eye View](https://mlsanity.com/paper/2403.12573/) (2024)
- [ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2308.13229/) (2023)
- [AeDet: Azimuth-invariant Multi-view 3D Object Detection](https://mlsanity.com/paper/2211.12501/) (2022)
- [Two-level Data Augmentation for Calibrated Multi-view Detection](https://mlsanity.com/paper/2210.10756/) (2022)
- [Bringing Generalization to Deep Multi-View Pedestrian Detection](https://mlsanity.com/paper/2109.12227/) (2021)
- [Multi-view Tracking Using Weakly Supervised Human Motion Prediction](https://mlsanity.com/paper/2210.10771/) (2022)
- [MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos](https://mlsanity.com/paper/2412.00692/) (2024)
- [Unsupervised Multi-view Pedestrian Detection](https://mlsanity.com/paper/2305.12457/) (2023)
- [Language-Driven Interactive Shadow Detection](https://mlsanity.com/paper/2408.08543/) (2024)
- [Voxelized 3D Feature Aggregation for Multiview Detection](https://mlsanity.com/paper/2112.03471/) (2021)
- [Enhanced Multi-View Pedestrian Detection Using Probabilistic Occupancy Volume](https://mlsanity.com/paper/2503.10982/) (2025)
- [Booster-SHOT: Boosting Stacked Homography Transformations for Multiview Pedestrian Detection with Attention](https://mlsanity.com/paper/2208.09211/) (2022)
- [From a Bird's Eye View to See: Joint Camera and Subject Registration without the Camera Calibration](https://mlsanity.com/paper/2212.09298/) (2022)
- [Multi-View People Detection in Large Scenes via Supervised View-Wise Contribution Weighting](https://mlsanity.com/paper/2405.19943/) (2024)
- [MITracker: Multi-View Integration for Visual Object Tracking](https://mlsanity.com/paper/2502.20111/) (2025)
- [DCHM: Depth-Consistent Human Modeling for Multiview Detection](https://mlsanity.com/paper/2507.14505/) (2025)
- [MVTrajecter: Multi-View Pedestrian Tracking with Trajectory Motion Cost and Trajectory Appearance Cost](https://mlsanity.com/paper/2509.01157/) (2025)
- [Adaptive Affinity for Associations in Multi-Target Multi-Camera Tracking](https://mlsanity.com/paper/2112.07664/) (2021)
- [Learning to Select Camera Views: Efficient Multiview Understanding at Few Glances](https://mlsanity.com/paper/2303.06145/) (2023)
- [Optimizing Camera Configurations for Multi-View Pedestrian Detection](https://mlsanity.com/paper/2312.02144/) (2023)
- [Multiview Detection with Cardboard Human Modeling](https://mlsanity.com/paper/2207.02013/) (2022)
- [Resource-Efficient Multiview Perception: Integrating Semantic Masking with Masked Autoencoders](https://mlsanity.com/paper/2410.04817/) (2024)
- [MVUDA: Unsupervised Domain Adaptation for Multi-view Pedestrian Detection](https://mlsanity.com/paper/2412.04117/) (2024)
- [MC-BEVRO: Multi-Camera Bird Eye View Road Occupancy Detection for Traffic Monitoring](https://mlsanity.com/paper/2502.11287/) (2025)
- [Attention-Aware Multi-View Pedestrian Tracking](https://mlsanity.com/paper/2504.03047/) (2025)
- [Semi-Supervised Multi-View Crowd Counting by Ranking Multi-View Fusion Models](https://mlsanity.com/paper/2512.16243/) (2025)

## Abstract (arXiv)

Multiview detection incorporates multiple camera views to deal with occlusions, and its central problem is multiview aggregation. Given feature map projections from multiple views onto a common ground plane, the state-of-the-art method addresses this problem via convolution, which applies the same calculation regardless of object locations. However, such translation-invariant behaviors might not be the best choice, as object features undergo various projection distortions according to their positions and cameras. In this paper, we propose a novel multiview detector, MVDeTr, that adopts a newly introduced shadow transformer to aggregate multiview information. Unlike convolutions, shadow transformer attends differently at different positions and cameras to deal with various shadow-like distortions. We propose an effective training scheme that includes a new view-coherent data augmentation method, which applies random augmentations while maintaining multiview consistency. On two multiview detection benchmarks, we report new state-of-the-art accuracy with the proposed system. Code is available at https://github.com/hou-yz/MVDeTr.