mlsanityData to 7 Oct 2026

Understanding the limitations of self-supervised learning for tabular anomaly detection

arXiv 2309.08374 ↗First posted 15 Sep 2023cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

While self-supervised learning has improved anomaly detection in computer vision and natural language processing, it is unclear whether tabular data can benefit from it. This paper explores the limitations of self-supervision for tabular anomaly detection. We conduct several experiments spanning various pretext tasks on 26 benchmark datasets to understand why this is the case. Our results confirm representations derived from self-supervision do not improve tabular anomaly detection performance compared to using the raw representations of the data. We show this is due to neural networks introducing irrelevant features, which reduces the effectiveness of anomaly detectors. However, we demonstrate that using a subspace of the neural network's representation can recover performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 41 of its 64 references that are in the map. First posted on arXiv 15 Sep 2023.

Builds on

41 of 64 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2018Deep contextualized word representationscited by 2,285
  8. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  10. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  12. 2016Colorful Image Colorizationcited by 1,113
29 more
  1. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  2. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  3. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  4. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  5. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  6. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
  7. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  8. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  9. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  10. 2019Deep Semi-Supervised Anomaly Detectioncited by 290
  11. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  12. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  13. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  14. 2023A Cookbook of Self-Supervised Learningcited by 248
  15. 2022ADBench: Anomaly Detection Benchmarkcited by 230
  16. 2020Classification-Based Anomaly Detection for General Datacited by 210
  17. 2020PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentationcited by 175
  18. 2021Adversarial Reciprocal Points Learning for Open Set Recognitioncited by 153
  19. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  20. 2020Deep Nearest Neighbor Anomaly Detectioncited by 103
  21. 2023White-Box Transformers via Sparse Rate Reductioncited by 77
  22. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  23. 2021Compressive Visual Representationscited by 41
  24. 2019Statistical Analysis of Nearest Neighbor Methods for Anomaly Detectioncited by 29
  25. 2020Why is the Mahalanobis Distance Effective for Anomaly Detection?cited by 28
  26. 2020Benchmarking Unsupervised Outlier Detection with Realistic Synthetic Datacited by 25
  27. 2022Anomaly Detection Requires Better Representationscited by 14
  28. 2022Self-Supervised Losses for One-Class Textual Anomaly Detectioncited by 5
  29. 2023pNNCLR: Stochastic Pseudo Neighborhoods for Contrastive Learning based Unsupervised Representation Learning Problemscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025HyPV-LEAD: Proactive Early-Warning of Cryptocurrency Anomalies through Data-Driven Structural-Temporal Modelingcited by 2
  2. 2026uLEAD-TabPFN: Uncertainty-aware Dependency-based Anomaly Detection with TabPFNcited by 1
  3. 2026LLM as Detector: An In-context Learning Approach for Tabular Anomaly Detectioncited by 0