mlsanityData to 14 Oct 2026

Detecting Semantic Backdoors in a Mystery Shopping Scenario

arXiv 2601.03805 ↗First posted 7 Jan 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Detecting semantic backdoors in classification models--where some classes can be activated by certain natural, but out-of-distribution inputs--is an important problem that has received relatively little attention. Semantic backdoors are significantly harder to detect than backdoors that are based on trigger patterns due to the lack of such clearly identifiable patterns. We tackle this problem under the assumption that the clean training dataset and the training recipe of the model are both known. These assumptions are motivated by a consumer protection scenario, in which the responsible authority performs mystery shopping to test a machine learning service provider. In this scenario, the authority uses the provider's resources and tools to train a model on a given dataset and tests whether the provider included a backdoor. In our proposed approach, the authority creates a reference model pool by training a small number of clean and poisoned models using trusted infrastructure, and calibrates a model distance threshold to identify clean models. We propose and experimentally analyze a number of approaches to compute model distances and we also test a scenario where the provider performs an adaptive attack to avoid detection. The most reliable method is based on requesting adversarial training from the provider. The model distance is best measured using a set of input samples generated by inverting the models in such a way as to maximize the distance from clean samples. With these settings, our method can often completely separate clean and poisoned models, and it proves to be superior to state-of-the-art backdoor detectors as well.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 67 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

44 of 67 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2016Wide Residual Networkscited by 3,070
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  10. 2017Deep Image Priorcited by 1,098
  11. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  12. 2018How To Backdoor Federated Learningcited by 765
32 more
  1. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  2. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 519
  3. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
  4. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
  5. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 425
  6. 2019Hidden Trigger Backdoor Attackscited by 371
  7. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 349
  8. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 324
  9. 2020Input-Aware Dynamic Backdoor Attackcited by 313
  10. 2019Label-Consistent Backdoor Attackscited by 296
  11. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  12. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 235
  13. 2021Improving Robustness using Generated Datacited by 234
  14. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 180
  15. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 151
  16. 2020Entangled Watermarks as a Defense against Model Extractioncited by 149
  17. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 146
  18. 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 130
  19. 2021Dataset Inference: Ownership Resolution in Machine Learningcited by 116
  20. 2019DAWN: Dynamic Adversarial Watermarking of Neural Networkscited by 100
  21. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 98
  22. 2023SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistencycited by 97
  23. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 95
  24. 2020Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Casescited by 95
  25. 2018Label Sanitization against Label Flipping Poisoning Attackscited by 73
  26. 2019IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundarycited by 70
  27. 2021Defending against Model Stealing via Verifying Embedded External Featurescited by 46
  28. 2021Topological Detection of Trojaned Neural Networkscited by 34
  29. 2021ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detectioncited by 33
  30. 2022Vulnerabilities of Deep Learning-Driven Semantic Communications to Backdoor (Trojan) Attackscited by 10
  31. 2023A semantic backdoor attack against Graph Convolutional Networkscited by 8
  32. 2024Backdoor Attacks and Defenses on Semantic-Symbol Reconstruction in Semantic Communicationscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Rethinking Backdoor Detection Evaluation for Language Models
  2. 2024An Invisible Backdoor Attack Based On Semantic Feature
  3. 2024Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks
  4. 2022Detecting Backdoors in Deep Text Classifiers
  5. 2023Backdoor Attacks for In-Context Learning with Language Models
  6. 2025Lie Detector: Unified Backdoor Detection via Cross-Examination Framework
  7. 2025DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited Data
  8. 2023BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection