# Sign-Symmetry Learning Rules are Robust Fine-Tuners

arXiv: https://arxiv.org/abs/2502.05925 · HTML: https://mlsanity.com/paper/2502.05925/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 9 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Adversarial Risk and the Dangers of Evaluating Against Weak Attacks](https://mlsanity.com/paper/1802.05666/) (2018)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation](https://mlsanity.com/paper/1407.7906/) (2014)
- [The Limitations of Adversarial Training and the Blind-Spot Attack](https://mlsanity.com/paper/1901.04684/) (2019)
- [How Important is Weight Symmetry in Backpropagation?](https://mlsanity.com/paper/1510.05067/) (2015)
- [Random feedback weights support learning in deep neural networks](https://mlsanity.com/paper/1411.0247/) (2014)
- [Understanding Synthetic Gradients and Decoupled Neural Interfaces](https://mlsanity.com/paper/1703.00522/) (2017)
- [Biologically Inspired Mechanisms for Adversarial Robustness](https://mlsanity.com/paper/2006.16427/) (2020)
- [Learning to solve the credit assignment problem](https://mlsanity.com/paper/1906.00889/) (2019)
- [Semantic-Aware Adversarial Training for Reliable Deep Hashing Retrieval](https://mlsanity.com/paper/2310.14637/) (2023)
- [HyP2 Loss: Beyond Hypersphere Metric Space for Multi-label Image Retrieval](https://mlsanity.com/paper/2208.06866/) (2022)
- [Deep Hashing via Householder Quantization](https://mlsanity.com/paper/2311.04207/) (2023)
- [Leveraging High-Resolution Features for Improved Deep Hashing-based Image Retrieval](https://mlsanity.com/paper/2403.13747/) (2024)

## Abstract (arXiv)

Backpropagation (BP) has long been the predominant method for training neural networks due to its effectiveness. However, numerous alternative approaches, broadly categorized under feedback alignment, have been proposed, many of which are motivated by the search for biologically plausible learning mechanisms. Despite their theoretical appeal, these methods have consistently underperformed compared to BP, leading to a decline in research interest. In this work, we revisit the role of such methods and explore how they can be integrated into standard neural network training pipelines. Specifically, we propose fine-tuning BP-pre-trained models using Sign-Symmetry learning rules and demonstrate that this approach not only maintains performance parity with BP but also enhances robustness. Through extensive experiments across multiple tasks and benchmarks, we establish the validity of our approach. Our findings introduce a novel perspective on neural network training and open new research directions for leveraging biologically inspired learning rules in deep learning.