mlsanityData to 14 Oct 2026

A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model

arXiv 2305.11244 ↗First posted 18 May 2023cs.CL cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

In this work, we explore Parameter-Efficient-Learning (PEL) techniques to repurpose a General-Purpose-Speech (GSM) model for Arabic dialect identification (ADI). Specifically, we investigate different setups to incorporate trainable features into a multi-layer encoder-decoder GSM formulation under frozen pre-trained settings. Our architecture includes residual adapter and model reprogramming (input-prompting). We design a token-level label mapping to condition the GSM for Arabic Dialect Identification (ADI). This is challenging due to the high variation in vocabulary and pronunciation among the numerous regional dialects. We achieve new state-of-the-art accuracy on the ADI-17 dataset by vanilla fine-tuning. We further reduce the training budgets with the PEL method, which performs within 1.86% accuracy to fine-tuning using only 2.5% of (extra) network trainable parameters. Our study demonstrates how to identify Arabic dialects using a small dataset and limited computation with open source code and pre-trained models.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

25 of 37 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  9. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  10. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  11. 2017Learning multiple visual domains with residual adapterscited by 557
  12. 2018Adversarial Reprogramming of Neural Networkscited by 116
13 more
  1. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  2. 20161.5 billion words Arabic Corpuscited by 35
  3. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 32
  4. 2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 31
  5. 2022AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Taskscited by 29
  6. 2018Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identificationcited by 23
  7. 2023From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognitioncited by 23
  8. 2022Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogrammingcited by 17
  9. 2022CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Modelscited by 13
  10. 2015Automatic Dialect Detection in Arabic Broadcast Speechcited by 5
  11. 2017MIT-QCRI Arabic Dialect Identification System for the 2017 Multi-Genre Broadcast Challengecited by 5
  12. 2020Transformer-based Arabic Dialect Identificationcited by 5
  13. 2022Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spottingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 1; 2026: 1.

8 papers.

  1. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  2. 2024NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Taskcited by 23
  3. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  4. 2023SpeechGen: Unlocking the Generative Power of Speech Language Models with Promptscited by 21
  5. 2023Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Expertscited by 17
  6. 2024Efficient Compression of Multitask Multilingual Speech Modelscited by 2
  7. 2025Dialect Identification Using Resource-Efficient Fine-Tuning Approachescited by 0
  8. 2026Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusioncited by 0