# Ptolemy: Architecture Support for Robust Deep Learning

arXiv: https://arxiv.org/abs/2008.09954 · HTML: https://mlsanity.com/paper/2008.09954/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2020.

## Built on by (most cited first, 12 of 12)

- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [Block-Skim: Efficient Question Answering for Transformer](https://mlsanity.com/paper/2112.08560/) (2021)
- [ScaleCert: Scalable Certified Defense against Adversarial Patches with Sparse Superficial Layers](https://mlsanity.com/paper/2110.14120/) (2021)
- [Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs](https://mlsanity.com/paper/2105.06512/) (2021)
- [2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency](https://mlsanity.com/paper/2109.05223/) (2021)
- [Real-Time Robust Video Object Detection System Against Physical-World Adversarial Attacks](https://mlsanity.com/paper/2208.09195/) (2022)
- [Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training](https://mlsanity.com/paper/2209.10778/) (2022)
- [Thales: Formulating and Estimating Architectural Vulnerability Factors for DNN Accelerators](https://mlsanity.com/paper/2212.02649/) (2022)
- [HASI: Hardware-Accelerated Stochastic Inference, A Defense Against Adversarial Machine Learning Attacks](https://mlsanity.com/paper/2106.05825/) (2021)
- [DNNShield: Dynamic Randomized Model Sparsification, A Defense Against Adversarial Machine Learning](https://mlsanity.com/paper/2208.00498/) (2022)
- [Characterizing Soft-Error Resiliency in Arm's Ethos-U55 Embedded Machine Learning Accelerator](https://mlsanity.com/paper/2404.09317/) (2024)

## Abstract (arXiv)

Deep learning is vulnerable to adversarial attacks, where carefully-crafted input perturbations could mislead a well-trained Deep Neural Network to produce incorrect results. Today's countermeasures to adversarial attacks either do not have capability to detect adversarial samples at inference time, or introduce prohibitively high overhead to be practical at inference time. We propose Ptolemy, an algorithm-architecture co-designed system that detects adversarial attacks at inference time with low overhead and high accuracy.We exploit the synergies between DNN inference and imperative program execution: an input to a DNN uniquely activates a set of neurons that contribute significantly to the inference output, analogous to the sequence of basic blocks exercised by an input in a conventional program. Critically, we observe that adversarial samples tend to activate distinctive paths from those of benign inputs. Leveraging this insight, we propose an adversarial sample detection framework, which uses canary paths generated from offline profiling to detect adversarial samples at runtime. The Ptolemy compiler along with the co-designed hardware enable efficient execution by exploiting the unique algorithmic characteristics. Extensive evaluations show that Ptolemy achieves higher or similar adversarial example detection accuracy than today's mechanisms with a much lower runtime (as low as 2%) overhead.