mlsanityData to 7 Oct 2026

Deep Learning using Linear Support Vector Machines

arXiv 1306.0239 ↗First posted 2 Jun 2013cs.LG stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

Recently, fully-connected and convolutional neural networks have been trained to achieve state-of-the-art performance on a wide variety of tasks such as speech recognition, image classification, natural language processing, and bioinformatics. For classification tasks, most of these "deep learning" models employ the softmax activation function for prediction and minimize cross-entropy loss. In this paper, we demonstrate a small but consistent advantage of replacing the softmax layer with a linear support vector machine. Learning minimizes a margin-based loss instead of the cross-entropy loss. While there have been various combinations of neural nets and SVMs in prior art, our results using L2-SVMs show that by simply replacing softmax with linear SVMs gives significant gains on popular deep learning datasets MNIST, CIFAR-10, and the ICML 2013 Representation Learning Workshop's face expression recognition challenge.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jun 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 6; 2023: 7; 2024: 5; 2025: 4; 2026: 3.

39 papers.

Sort:

  1. 2021Facial Expression Recognition with Visual Transformers and Attentional Selective Fusioncited by 49
  2. 2021Facial Emotion Recognition: State of the Art Performance on FER2013cited by 20
  3. 2021On the Equivalence between Neural Network and Support Vector Machinecited by 17
  4. 2022Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Datacited by 11
  5. 2021Teacher-Student Training and Triplet Loss to Reduce the Effect of Drastic Face Occlusioncited by 9
  6. 2023FER-former: Multi-modal Transformer for Facial Expression Recognitioncited by 8
  7. 2021Recent Advances in Large Margin Learningcited by 7
  8. 2021Face Trees for Expression Recognitioncited by 6
  9. 2022Glo-In-One: Holistic Glomerular Detection, Segmentation, and Lesion Characterization with Large-scale Web Image Miningcited by 6
  10. 2021New SAR target recognition based on YOLO and very deep multi-canonical correlation analysiscited by 5
  11. 2022FaceTopoNet: Facial Expression Recognition using Face Topology Learningcited by 4
  12. 2024SegNet: A Segmented Deep Learning based Convolutional Neural Network Approach for Drones Wildfire Detectioncited by 4
27 more
  1. 2021Regional Attention Network (RAN) for Head Pose and Fine-grained Gesture Recognitioncited by 3
  2. 2024Self-Supervised Interpretable End-to-End Learning via Latent Functional Modularitycited by 3
  3. 2021Quantum Machine Learning with HQC Architectures using non-Classically Simulable Feature Mapscited by 2
  4. 2023Kernel function impact on convolutional neural networkscited by 2
  5. 2024A Mirror Descent-Based Algorithm for Corruption-Tolerant Distributed Gradient Descentcited by 2
  6. 2025Exploring Kernel Transformations for Implicit Neural Representationscited by 2
  7. 2021MAFER: a Multi-resolution Approach to Facial Expression Recognitioncited by 1
  8. 2021Recognizing Facial Expressions in the Wild using Multi-Architectural Representations based Ensemble Learning with Distillationcited by 1
  9. 2023Algorithms for the Training of Neural Support Vector Machinescited by 1
  10. 2026Large-Margin Hyperdimensional Computing: A Learning-Theoretical Perspectivecited by 1
  11. 2021Classification of COVID-19 X-ray Images Using a Combination of Deep and Handcrafted Featurescited by 0
  12. 2021A Multi-resolution Approach to Expression Recognition in the Wildcited by 0
  13. 2021A Novel Hybrid Deep Learning Approach for Non-Intrusive Load Monitoring of Residential Appliance Based on Long Short Term Memory and Convolutional Neural Networkscited by 0
  14. 2022Deep Low-Density Separation for Semi-Supervised Classificationcited by 0
  15. 2022Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networkscited by 0
  16. 2022Finetuning for Sarcasm Detection with a Pruned Datasetcited by 0
  17. 2023Neural networks for learning personality traits from natural languagecited by 0
  18. 2023Maximum margin learning of t-SPNs for cell classification with filtered inputcited by 0
  19. 2023Automated Solubility Analysis System and Method Using Computer Vision and Machine Learningcited by 0
  20. 2023On Neural Quantum Support Vector Machinescited by 0
  21. 2024Feature Network Methods in Machine Learning and Applicationscited by 0
  22. 2024Deep Support Vectorscited by 0
  23. 2025Data Complexity Measures for Quantum Circuits Architecture Recommendationcited by 0
  24. 2025A Novel Frame Identification and Synchronization Technique for Smartphone Visible Light Communication Systems Based on Convolutional Neural Networkscited by 0
  25. 2025Transformer-Driven Multimodal Fusion for Explainable Suspiciousness Estimation in Visual Surveillancecited by 0
  26. 2026Facial Expression Recognition Using Residual Masking Networkcited by 0
  27. 2026TileNet: Tile-Based CNN-SVM Architecture for Autonomous Unmanned Aerial Systems Inspection of Flat Roofscited by 0