# ACIT: Attention-Guided Cross-Modal Interaction Transformer for Pedestrian Crossing Intention Prediction

arXiv: https://arxiv.org/abs/2511.20020 · HTML: https://mlsanity.com/paper/2511.20020/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs](https://mlsanity.com/paper/2005.06582/) (2020)
- [Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention](https://mlsanity.com/paper/2104.05485/) (2021)
- [Bifold and Semantic Reasoning for Pedestrian Behavior Prediction](https://mlsanity.com/paper/2012.03298/) (2020)
- [Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet](https://mlsanity.com/paper/1904.09862/) (2019)
- [Agreeing to Cross: How Drivers and Pedestrians Communicate](https://mlsanity.com/paper/1702.03555/) (2017)
- [Multi-Modal Hybrid Architecture for Pedestrian Action Prediction](https://mlsanity.com/paper/2012.00514/) (2020)
- [Intention Recognition of Pedestrians and Cyclists by 2D Pose Estimation](https://mlsanity.com/paper/1910.03858/) (2019)
- [FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network](https://mlsanity.com/paper/2005.07796/) (2020)
- [IntFormer: Predicting pedestrian intention with the aid of the Transformer architecture](https://mlsanity.com/paper/2105.08647/) (2021)
- [TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction](https://mlsanity.com/paper/2109.00953/) (2021)
- [Deep Virtual-to-Real Distillation for Pedestrian Crossing Prediction](https://mlsanity.com/paper/2211.00856/) (2022)

## Built on by (most cited first, 3 of 3)

- [Adaptive Cross-Modal Fusion with Sparse Attention for Pedestrian Crossing Intention Prediction](https://mlsanity.com/paper/2607.12293/) (2026)
- [Detecting Phone-Induced Pedestrian Distraction via a Multimodal Fusion Transformer](https://mlsanity.com/paper/2609.23507/) (2026)
- [Lightweight Pedestrian Head-Orientation Recognition Network for Safe Pedestrian-Vehicle Interaction](https://mlsanity.com/paper/2609.24193/) (2026)

## Abstract (arXiv)

Predicting pedestrian crossing intention is crucial for autonomous vehicles to prevent pedestrian-related collisions. However, effectively extracting and integrating complementary cues from different types of data remains one of the major challenges. This paper proposes an attention-guided cross-modal interaction Transformer (ACIT) for pedestrian crossing intention prediction. ACIT leverages six visual and motion modalities, which are grouped into three interaction pairs: (1) Global semantic map and global optical flow, (2) Local RGB image and local optical flow, and (3) Ego-vehicle speed and pedestrian's bounding box. Within each visual interaction pair, a dual-path attention mechanism enhances salient regions within the primary modality through intra-modal self-attention and facilitates deep interactions with the auxiliary modality (i.e., optical flow) via optical flow-guided attention. Within the motion interaction pair, cross-modal attention is employed to model the cross-modal dynamics, enabling the effective extraction of complementary motion features. Beyond pairwise interactions, a multi-modal feature fusion module further facilitates cross-modal interactions at each time step. Furthermore, a Transformer-based temporal feature aggregation module is introduced to capture sequential dependencies. Experimental results demonstrate that ACIT outperforms state-of-the-art methods, achieving accuracy rates of 70% and 89% on the JAADbeh and JAADall datasets, respectively. Extensive ablation studies are further conducted to investigate the contribution of different modules of ACIT.