mlsanityData to 7 Oct 2026

Voice2Series: Reprogramming Acoustic Models for Time Series Classification

arXiv 2106.09296 ↗First posted 17 Jun 2021cs.LG cs.AI cs.NEcited by 91

Not on the map. Why?

Abstract (arXiv)

Learning to classify time series with limited data is a practical yet challenging problem. Current methods are primarily based on hand-designed feature extraction rules or domain-specific data augmentation. Motivated by the advances in deep speech processing models and the fact that voice data are univariate temporal signals, in this paper, we propose Voice2Series (V2S), a novel end-to-end approach that reprograms acoustic models for time series classification, through input transformation learning and output label mapping. Leveraging the representation learning power of a large-scale pre-trained speech processing model, on 30 different time series tasks we show that V2S performs competitive results on 19 time series classification tasks. We further provide a theoretical justification of V2S by proving its population risk is upper bounded by the source risk and a Wasserstein distance accounting for feature alignment via reprogramming. Our results offer new and effective means to time series classification.

Key facts. Cited by 91 papers in the map (data through 7 Oct 2026). Builds on 34 of its 61 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

34 of 61 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2018Computational Optimal Transportcited by 765
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  9. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  10. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  11. 2018Deep learning for time series classification: a reviewcited by 575
  12. 2018The UCR Time Series Archivecited by 421
22 more
  1. 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 358
  2. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  3. 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
  4. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 160
  5. 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 136
  6. 2018Adversarial Reprogramming of Neural Networkscited by 115
  7. 2020Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resourcescited by 83
  8. 2018Transfer learning for time series classificationcited by 60
  9. 2017Sliced Wasserstein Distance for Learning Gaussian Mixture Modelscited by 59
  10. 2020Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognitioncited by 56
  11. 2020Acoustic scene classification in DCASE 2020 Challenge: generalization across devices and low complexity solutionscited by 48
  12. 2018A neural attention model for speech command recognitioncited by 44
  13. 2017English Conversational Telephone Speech Recognition by Humans and Machinescited by 40
  14. 2019MINA: Multilevel Knowledge-Guided Attention for Modeling Electrocardiography Signalscited by 21
  15. 2019ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classificationcited by 20
  16. 2020Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancementcited by 18
  17. 2020Device-Robust Acoustic Scene Classification Based on Two-Stage Categorization and Data Augmentationcited by 13
  18. 2020Reprogramming Language Models for Molecular Representation Learningcited by 13
  19. 2021PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classificationcited by 10
  20. 2020A Two-Stage Approach to Device-Robust Acoustic Scene Classificationcited by 9
  21. 2019Enhancing Sound Texture in CNN-Based Acoustic Scene Classificationcited by 6
  22. 2017Kapre: On-GPU Audio Preprocessing Layers for a Quick Implementation of Deep Neural Network Models with Kerascited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 19; 2023: 28; 2024: 24; 2025: 13; 2026: 4.

91 papers.

Sort:

By area:

  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  2. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  3. 2022Transformers in Time Series: A Surveycited by 489
  4. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  5. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  6. 2023Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Promptingcited by 71
  7. 2023Deep Learning for Time Series Classification and Extrinsic Regression: A Current Surveycited by 68
  8. 2022Model Reprogramming: Resource-Efficient Cross-Domain Machine Learningcited by 64
  9. 2021The Dawn of Quantum Natural Language Processingcited by 60
  10. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  11. 2022Evaluation of post-hoc interpretability methods in time-series classificationcited by 53
  12. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
38 more
  1. 2023Improving Position Encoding of Transformers for Multivariate Time Series Classificationcited by 44
  2. 2022Data Augmentation techniques in time series domain: A survey and taxonomycited by 41
  3. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  4. 2022TranSpeech: Speech-to-Speech Translation With Bilateral Perturbationcited by 34
  5. 2022Visual Prompting for Adversarial Robustnesscited by 33
  6. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 31
  7. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 30
  8. 2022AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Taskscited by 29
  9. 2022Watermarking for Out-of-distribution Detectioncited by 28
  10. 2022Exploring Efficient-tuning Methods in Self-supervised Speech Modelscited by 26
  11. 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
  12. 2023From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognitioncited by 23
  13. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  14. 2022Uncertainty-DTW for Time Series and Sequencescited by 21
  15. 2023SpeechGen: Unlocking the Generative Power of Speech Language Models with Promptscited by 21
  16. 2024Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotationcited by 21
  17. 2022Reprogramming Pretrained Language Models for Antibody Sequence Infillingcited by 20
  18. 2025REFINE: Inversion-Free Backdoor Defense via Model Reprogrammingcited by 20
  19. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  20. 2022SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Taskscited by 19
  21. 2023Text-Visual Prompting for Efficient 2D Temporal Video Groundingcited by 18
  22. 2022Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogrammingcited by 17
  23. 2023Convolutional Visual Prompt for Robust Visual Perceptioncited by 17
  24. 2023Exploring the Benefits of Visual Prompting in Differential Privacycited by 17
  25. 2024Algorithmic Capabilities of Random Transformerscited by 17
  26. 2024Sample-specific Masks for Visual Reprogramming-based Promptingcited by 16
  27. 2022Reservoir Computing via Quantum Recurrent Neural Networkscited by 14
  28. 2021Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Differencecited by 13
  29. 2022CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Modelscited by 13
  30. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  31. 2023Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognitioncited by 13
  32. 2024Bayesian-guided Label Mapping for Visual Reprogrammingcited by 12
  33. 2022Holistic Adversarial Robustness of Deep Learning Modelscited by 11
  34. 2024Only the Curve Shape Matters: Training Foundation Models for Zero-Shot Multivariate Time Series Forecasting through Next Curve Shape Predictioncited by 11
  35. 2024Reprogramming Pretrained Target-Specific Diffusion Models for Dual-Target Drug Designcited by 11
  36. 2022Adversarial Reprogramming Revisitedcited by 10
  37. 2023How to Estimate Model Transferability of Pre-Trained Speech Models?cited by 10
  38. 2023Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classificationcited by 10

50 most cited of 91 All 91 (JSON)