mlsanityData to 7 Oct 2026

Leveraging Expert Models for Training Deep Neural Networks in Scarce Data Domains: Application to Offline Handwritten Signature Verification

arXiv 2308.01136 ↗First posted 2 Aug 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper introduces a novel approach to leverage the knowledge of existing expert models for training new Convolutional Neural Networks, on domains where task-specific data are limited or unavailable. The presented scheme is applied in offline handwritten signature verification (OffSV) which, akin to other biometric applications, suffers from inherent data limitations due to regulatory restrictions. The proposed Student-Teacher (S-T) configuration utilizes feature-based knowledge distillation (FKD), combining graph-based similarity for local activations with global similarity measures to supervise student's training, using only handwritten text data. Remarkably, the models trained using this technique exhibit comparable, if not superior, performance to the teacher model across three popular signature datasets. More importantly, these results are attained without employing any signatures during the feature extraction training process. This study demonstrates the efficacy of leveraging existing expert models to overcome data scarcity challenges in OffSV and potentially other related domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 98 references that are in the map. First posted on arXiv 2 Aug 2023.

Builds on

32 of 98 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  10. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  11. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  12. 2020Knowledge Distillation: A Surveycited by 1,578
20 more
  1. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  2. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  3. 2013Do Deep Nets Really Need to be Deep?cited by 466
  4. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
  5. 2017Beyond triplet loss: a deep quadruplet network for person re-identificationcited by 172
  6. 2017SigNet: Convolutional Siamese Network for Writer Independent Offline Signature Verificationcited by 31
  7. 2017Learning Features for Offline Handwritten Signature Verification using Deep Convolutional Neural Networkscited by 20
  8. 2015Offline Handwritten Signature Verification - Literature Reviewcited by 15
  9. 2019Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verificationcited by 9
  10. 2022SURDS: Self-Supervised Attention-guided Reconstruction and Dual Triplet Loss for Writer Independent Offline Signature Verificationcited by 8
  11. 2022SWIS: Self-Supervised Representation Learning For Writer Independent Offline Signature Verificationcited by 7
  12. 2018Fixed-sized representation learning from Offline Handwritten Signatures of different sizescited by 4
  13. 2020Offline Signature Verification on Real-World Documentscited by 4
  14. 2017Signature Verification Approach using Fusion of Hybrid Texture Featurescited by 3
  15. 2018A comprehensive study of sparse representation techniques for offline signature verificationcited by 3
  16. 2020Intrapersonal Parameter Optimization for Offline Handwritten Signature Augmentationcited by 3
  17. 2019Learning Representations from Persian Handwriting for Offline Signature Verification, a Deep Transfer Learning Approachcited by 2
  18. 2019Multi-Representational Learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNscited by 2
  19. 2019Active Transfer Learning for Persian Offline Signature Verificationcited by 1
  20. 2020A white-box analysis on the writer-independent dichotomy transformation applied to offline handwritten signature verificationcited by 1

Built on by

Not cited yet.