COMPASS: Fusion-Matched Supervision for Missing-Modality Human Sensing
Not on the map. Why?
Abstract (arXiv)
Multimodal human activity recognition (HAR) and human pose estimation (HPE) must cope with modalities missing at inference. Completions generated for a fixed fusion model must preserve the fusion readout: what fusion uses from each modality. We propose COMPASS, a completion-and-fusion framework in which each modality occupies a fixed slot filled by an observed representation or a completion inferred from available inputs. Its core principle, fusion-matched supervision, supervises completions with same-sample real targets at readout granularity: token means for HAR, where fusion averages tokens, and per-joint representations for HPE, where fusion preserves joint structure. Trained with complete multimodal samples, COMPASS improves subset-averaged performance over the strongest baselines on XRF55 and MM-Fi for HAR and on MM-Fi for HPE. At fixed architecture, fusion-matched supervision outperforms full-token and centered-token matching in HAR, and joint-mean matching and no matching in HPE. The code is available at: https://github.com/haowangcoder/COMPASS.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
16 of 22 references.
References · 16
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
- 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
- 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
- 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 163
- 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
- 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
- 2023MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensingcited by 79
- 2022Towards Good Practices for Missing Modality Robust Action Recognitioncited by 54
- 2024Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Modelscited by 20
4 more
- 2023Towards Robust Multimodal Prompting With Missing Modalitiescited by 16
- 2024X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensingcited by 13
- 2025Robust Multimodal Learning via Cross-Modal Proxy Tokenscited by 2
- 2026Diverse via bounded Agreement: Geometric Regularization for Multimodal Fusioncited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.