RayS: A Ray Searching Method for Hard-label Adversarial Attack
Not on the map. Why?
Abstract (arXiv)
Deep neural networks are vulnerable to adversarial attacks. Among different attack settings, the most challenging yet the most practical one is the hard-label setting where the attacker only has access to the hard-label output (prediction label) of the target model. Previous attempts are neither effective enough in terms of attack success rate nor efficient enough in terms of query complexity under the widely used L_∞ norm threat model. In this paper, we present the Ray Searching attack (RayS), which greatly improves the hard-label attack effectiveness as well as efficiency. Unlike previous works, we reformulate the continuous problem of finding the closest decision boundary into a discrete problem that does not require any zeroth-order gradient estimation. In the meantime, all unnecessary searches are eliminated via a fast check step. This significantly reduces the number of queries needed for our hard-label attack. Moreover, interestingly, we found that the proposed RayS attack can also be used as a sanity check for possible "falsely robust" models. On several recently proposed defenses that claim to achieve the state-of-the-art robust accuracy, our attack method demonstrates that the current white-box/black-box attacks could still give a false sense of security and the robust accuracy drop between the most popular PGD attack and RayS attack could be as large as 28%. We believe that our proposed RayS attack could help identify falsely robust models that beat most white-box/black-box attacks.
Key facts. Cited by 94 papers in the map (data through 13 Oct 2026). First posted on arXiv 23 Jun 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 19; 2022: 19; 2023: 25; 2024: 14; 2025: 12; 2026: 5.
94 papers.
Sort:
By area:
- 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
- 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 110
- 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 76
- 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
- 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
- 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
- 2021Attacking Adversarial Attacks as A Defensecited by 31
- 2021A Review of Adversarial Attack and Defense for Classification Methodscited by 29
- 2023Generalizable Black-Box Adversarial Attack with Meta Learningcited by 29
- 2022One-shot Neural Backdoor Erasing via Adversarial Weight Maskingcited by 28
- 2023Randomized Adversarial Training via Taylor Expansioncited by 28
- 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attackcited by 28
38 more
- 2022Removing Batch Normalization Boosts Adversarial Trainingcited by 24
- 2024ADBM: Adversarial diffusion bridge model for reliable adversarial purificationcited by 24
- 2022Scaling Adversarial Training to Large Perturbation Boundscited by 20
- 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 20
- 2023CGBA: Curvature-aware Geometric Black-box Attackcited by 20
- 2023A Survey of Robustness and Safety of 2D and 3D Deep Learning Models Against Adversarial Attackscited by 19
- 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agentscited by 19
- 2022Recognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networkscited by 18
- 2021Back in Black: A Comparative Evaluation of Recent State-Of-The-Art Black-Box Attackscited by 16
- 2021Two Coupled Rejection Metrics Can Tell Adversarial Examples Apartcited by 14
- 2021Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfittingcited by 14
- 2023Fast Adversarial Training with Smooth Convergencecited by 14
- 2022Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Modelscited by 13
- 2023Evading Black-box Classifiers Without Breaking Eggscited by 13
- 2023Mitigating Accuracy-Robustness Trade-off via Balanced Multi-Teacher Adversarial Distillationcited by 13
- 2023Boosting Black-box Attack to Deep Neural Networks with Conditional Diffusion Modelscited by 12
- 2021Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attackscited by 11
- 2022Part-Based Models Improve Adversarial Robustnesscited by 11
- 2021Improving Model Robustness with Latent Distribution Locally and Globallycited by 9
- 2021MedRDF: A Robust and Retrain-Less Diagnostic Framework for Medical Pretrained Models Against Adversarial Attackcited by 9
- 2023Robust Spatiotemporal Traffic Forecasting with Reinforced Dynamic Adversarial Trainingcited by 9
- 2025Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradientscited by 9
- 2021Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimationcited by 8
- 2022Adversarial Purification with the Manifold Hypothesiscited by 8
- 2021Rethinking Image-Scaling Attacks: The Interplay Between Vulnerabilities in Machine Learning Systemscited by 7
- 2021Performance Evaluation of Adversarial Attacks: Discrepancies and Solutionscited by 7
- 2022Learnability Lock: Authorized Learnability Control Through Adversarial Invertible Transformationscited by 7
- 2022Randomized Smoothing under Attack: How Good is it in Pratice?cited by 7
- 2022InfoAT: Improving Adversarial Training Using the Information Bottleneck Principlecited by 7
- 2022Adversarial Framework with Certified Robustness for Time-Series Domain via Statistical Featurescited by 7
- 2022Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attackcited by 7
- 2024Boosting Adversarial Transferability with Low-Cost Optimization via Maximin Expected Flatnesscited by 7
- 2024ADBA:Approximation Decision Boundary Approach for Black-Box Adversarial Attackscited by 7
- 2024AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuningcited by 7
- 2021Mixture of Robust Experts (MoRE):A Robust Denoising Method towards multiple perturbationscited by 6
- 2023SoK: Pitfalls in Evaluating Black-Box Attackscited by 6
- 2023Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidencecited by 5
- 2023Towards Robust and Accurate Visual Promptingcited by 5
50 most cited of 94 All 94 (JSON)
- 2026Signal-based Model Access Risk Analysis for AI System Operations Security17 Jul 2026
- 2026Latent Geometric Chords for Query-Efficient Decision-Based Adversarial Attacks29 May 2026
- 2026Low-Cost Hard-Label Adversarial Attack with Theoretical Foundations17 Jan 2026
- 2026Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models17 Jan 2026
- 2026SAFE: Secure and Accurate Federated Learning for Privacy-Preserving Brain-Computer Interfaces9 Jan 2026
- 2025Improving the Convergence Rate of Ray Search Optimization for Query-Efficient Hard-Label Attacks24 Dec 2025
- 2025Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation11 Dec 2025
- 2025FBA2D: Frequency-based Black-box Attack for AI-generated Image Detection10 Dec 2025
- 2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Security20 Nov 2025
- 2025Sequence-Preserving Dual-FoV Defense for Traffic Sign and Light Recognition in Autonomous Vehicles3 Oct 2025
- 2025Latent Danger Zone: Distilling Unified Attention for Cross-Architecture Black-box Attacks23 Sep 2025
- 2025DASH: A Meta-Attack Framework for Synthesizing Effective and Stealthy Adversarial Examples18 Aug 2025
38 more
- 2025Boosting Ray Search Procedure of Hard-label Attacks with Transfer-based Priors23 Jul 2025
- 2025Rewriting the Budget: A General Framework for Black-Box Attacks Under Cost Asymmetry7 Jun 2025
- 2025Adversarial Examples in Environment Perception for Automated Driving (Review)11 Apr 2025
- 2025Task-Agnostic Attacks Against Vision Foundation Models5 Mar 2025
- 2025Towards Effective and Sparse Adversarial Attack on Spiking Neural Networks via Breaking Invisible Surrogate Gradients5 Mar 2025
- 2024RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses30 Dec 2024
- 2024Interpretability-Guided Test-Time Adversarial Defense23 Sep 2024
- 2024ViTGuard: Attention-aware Detection against Adversarial Examples for Vision Transformer20 Sep 2024
- 2024Query-Efficient Video Adversarial Attack with Stylized Logo on Service Computing22 Aug 2024
- 2024AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning4 Aug 2024
- 2024ADBM: Adversarial diffusion bridge model for reliable adversarial purification1 Aug 2024
- 2024Evaluating the Adversarial Robustness of Semantic Segmentation: Trying Harder Pays Off12 Jul 2024
- 2024Rethinking the Threat and Accessibility of Adversarial Attacks against Face Recognition Systems11 Jul 2024
- 2024ProFeAT: Projected Feature Adversarial Training for Self-Supervised Learning of Robust Representations9 Jun 2024
- 2024ADBA:Approximation Decision Boundary Approach for Black-Box Adversarial Attacks7 Jun 2024
- 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents30 May 2024
- 2024Boosting Adversarial Transferability with Low-Cost Optimization via Maximin Expected Flatness25 May 2024
- 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack25 May 2024
- 2024ADAPT to Robustify Prompt Tuning Vision Transformers19 Mar 2024
- 2023DTA: Distribution Transform-based Attack for Query-Limited Scenario12 Dec 2023
- 2023Rethinking PGD Attack: Is Sign Function Necessary?3 Dec 2023
- 2023Adversarial Robust Memory-Based Continual Learner29 Nov 2023
- 2023Towards Robust and Accurate Visual Prompting18 Nov 2023
- 2023Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing15 Nov 2023
- 2023SoK: Pitfalls in Evaluating Black-Box Attacks26 Oct 2023
- 2023Exploring the Adversarial Robustness of Face Forgery Detection with Decision-based Black-box Attacks18 Oct 2023
- 2023Boosting Black-box Attack to Deep Neural Networks with Conditional Diffusion Models11 Oct 2023
- 2023A Survey of Robustness and Safety of 2D and 3D Deep Learning Models Against Adversarial Attacks1 Oct 2023
- 2023Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks1 Oct 2023
- 2023Fast Adversarial Training with Smooth Convergence24 Aug 2023
- 2023SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks23 Aug 2023
- 2023CGBA: Curvature-aware Geometric Black-box Attack6 Aug 2023
- 2023Mitigating Accuracy-Robustness Trade-off via Balanced Multi-Teacher Adversarial Distillation28 Jun 2023
- 2023On Evaluating the Adversarial Robustness of Semantic Segmentation Models25 Jun 2023
- 2023Robust Spatiotemporal Traffic Forecasting with Reinforced Dynamic Adversarial Training25 Jun 2023
- 2023Evading Black-box Classifiers Without Breaking Eggs5 Jun 2023
- 2023Enhance Diffusion to Improve Robust Generalization5 Jun 2023
- 2023Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidence10 Apr 2023
50 newest of 94 All 94 (JSON)
- 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 76
- 2024ADBM: Adversarial diffusion bridge model for reliable adversarial purificationcited by 24
- 2023Boosting Black-box Attack to Deep Neural Networks with Conditional Diffusion Modelscited by 12
- 2025Latent Danger Zone: Distilling Unified Attention for Cross-Architecture Black-box Attackscited by 0
- 2025Latent Danger Zone: Distilling Unified Attention for Cross-Architecture Black-box Attacks23 Sep 2025
- 2024ADBM: Adversarial diffusion bridge model for reliable adversarial purification1 Aug 2024
- 2023Boosting Black-box Attack to Deep Neural Networks with Conditional Diffusion Models11 Oct 2023
- 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion29 Mar 2023
- 2023Fast Certification of Vision-Language Models Using Incremental Randomized Smoothingcited by 4
- 2026Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Modelscited by 4
- 2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Securitycited by 2
- 2026Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models17 Jan 2026
- 2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Security20 Nov 2025
- 2023Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing15 Nov 2023
- 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
- 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 110
- 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
- 2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
- 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
- 2021Attacking Adversarial Attacks as A Defensecited by 31
- 2021A Review of Adversarial Attack and Defense for Classification Methodscited by 29
- 2023Generalizable Black-Box Adversarial Attack with Meta Learningcited by 29
- 2022One-shot Neural Backdoor Erasing via Adversarial Weight Maskingcited by 28
- 2023Randomized Adversarial Training via Taylor Expansioncited by 28
- 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attackcited by 28
- 2022Removing Batch Normalization Boosts Adversarial Trainingcited by 24
12 most cited of 86 All 86 (JSON)
- 2026Signal-based Model Access Risk Analysis for AI System Operations Security17 Jul 2026
- 2026Latent Geometric Chords for Query-Efficient Decision-Based Adversarial Attacks29 May 2026
- 2026Low-Cost Hard-Label Adversarial Attack with Theoretical Foundations17 Jan 2026
- 2026SAFE: Secure and Accurate Federated Learning for Privacy-Preserving Brain-Computer Interfaces9 Jan 2026
- 2025Improving the Convergence Rate of Ray Search Optimization for Query-Efficient Hard-Label Attacks24 Dec 2025
- 2025Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation11 Dec 2025
- 2025FBA2D: Frequency-based Black-box Attack for AI-generated Image Detection10 Dec 2025
- 2025Sequence-Preserving Dual-FoV Defense for Traffic Sign and Light Recognition in Autonomous Vehicles3 Oct 2025
- 2025DASH: A Meta-Attack Framework for Synthesizing Effective and Stealthy Adversarial Examples18 Aug 2025
- 2025Boosting Ray Search Procedure of Hard-label Attacks with Transfer-based Priors23 Jul 2025
- 2025Rewriting the Budget: A General Framework for Black-Box Attacks Under Cost Asymmetry7 Jun 2025
- 2025Adversarial Examples in Environment Perception for Automated Driving (Review)11 Apr 2025
12 newest of 86 All 86 (JSON)