mlsanityData to 14 Oct 2026

SPAR: Self-supervised Placement-Aware Representation Learning for Distributed Sensing

arXiv 2505.16936 ↗First posted 22 May 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present SPAR, a framework for self-supervised placement-aware representation learning in distributed sensing. Distributed sensing spans applications where multiple spatially distributed and multimodal sensors jointly observe an environment, from vehicle monitoring to human activity recognition and earthquake localization. A central challenge shared by this wide spectrum of applications is that observed signals are inseparably shaped by sensor placements, including their spatial locations and structural characteristics. However, existing pretraining methods remain largely placement-agnostic. SPAR addresses this gap through a unifying principle: the duality between signals and positions. Guided by this principle, SPAR introduces spatial and structural positional embeddings together with dual reconstruction objectives, explicitly modeling how observing positions and observed signals shape each other. Placement is thus treated not as auxiliary metadata but as intrinsic to representation learning. SPAR is theoretically supported by analyses from information theory and occlusion-invariant learning. Extensive experiments on three real-world datasets show that SPAR achieves superior robustness and generalization across various modalities, placements, and downstream tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 66 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

44 of 66 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  10. 2019Contrastive Multiview Codingcited by 1,301
  11. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  12. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
32 more
  1. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  2. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  3. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  4. 2023A decoder-only foundation model for time-series forecastingcited by 692
  5. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  6. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  7. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  8. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  9. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
  10. 2022Masked Autoencoders that Listencited by 273
  11. 2023TimeGPT-1cited by 158
  12. 2022Contrastive Audio-Visual Masked Autoencodercited by 146
  13. 2019StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understandingcited by 109
  14. 2018Learning Image Representations by Completing Damaged Jigsaw Puzzlescited by 66
  15. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  16. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  17. 2022Understanding Masked Image Modeling via Learning Occlusion Invariant Featurecited by 50
  18. 2023GeoMAE: Masked Geometric Target Prediction for Self-supervised Point Cloud Pre-Trainingcited by 38
  19. 2023FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Spacecited by 33
  20. 2017DeepPermNet: Visual Permutation Learningcited by 32
  21. 2022SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognitioncited by 29
  22. 2023USat: A Unified Self-Supervised Encoder for Multi-Sensor Satellite Imagerycited by 28
  23. 2024LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Tracescited by 28
  24. 2023Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecastingcited by 27
  25. 2020SLM: Learning a Discourse Language Representation with Sentence Unshufflingcited by 24
  26. 2022Position Prediction as an Effective Pretraining Strategycited by 24
  27. 2023SS-MAE: Spatial-Spectral Masked Auto-Encoder for Multi-Source Remote Sensing Image Classificationcited by 24
  28. 2022Location-Aware Self-Supervised Transformers for Semantic Segmentationcited by 20
  29. 2024IoT-LM: Large Multisensory Language Models for the Internet of Thingscited by 14
  30. 2024Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentationcited by 6
  31. 2024MMBind: Unleashing the Potential of Distributed and Heterogeneous Data for Multimodal Learning in IoTcited by 6
  32. 2024Order-Based Pre-training Strategies for Procedural Text Understandingcited by 2

Built on by

Not cited yet.