mlsanityData to 7 Oct 2026

Scaling up masked audio encoder learning for general audio classification

arXiv 2406.06992 ↗First posted 11 Jun 2024cs.SD eess.AScited by 43

Not on the map. Why?

Abstract (arXiv)

Despite progress in audio classification, a generalization gap remains between speech and other sound domains, such as environmental sounds and music. Models trained for speech tasks often fail to perform well on environmental or musical audio tasks, and vice versa. While self-supervised (SSL) audio representations offer an alternative, there has been limited exploration of scaling both model and dataset sizes for SSL-based general audio classification. We introduce Dasheng, a simple SSL audio encoder, based on the efficient masked autoencoder framework. Trained with 1.2 billion parameters on 272,356 hours of diverse audio, Dasheng obtains significant performance gains on the HEAR benchmark. It outperforms previous works on CREMA-D, LibriCount, Speech Commands, VoxLingua, and competes well in music and environment classification. Dasheng features inherently contain rich speech, music, and environmental information, as shown in nearest-neighbor classification experiments. Code is available https://github.com/richermans/dasheng/.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jun 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 16; 2026: 25.

43 papers.

Sort:

By area:

  1. 2024Foundation Models for Music: A Surveycited by 34
  2. 2025Audio-JEPA: Joint-Embedding Predictive Architecture for Audio Representation Learningcited by 17
  3. 2025GLAP: General contrastive audio-text pretraining across domains and languagescited by 10
  4. 2026Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scalecited by 10
  5. 2025Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classificationcited by 7
  6. 2025MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Taskscited by 6
  7. 2025MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learningcited by 5
  8. 2025FlexSED: Towards Open-Vocabulary Sound Event Detectioncited by 4
  9. 2025Revisiting Audio-language Pretraining for Learning General-purpose Audio Representationcited by 3
  10. 2026Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audiocited by 3
  11. 2025FISHER: A Foundation Model for Multi-Modal Industrial Signal Comprehensive Representationcited by 2
  12. 2026SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Modelscited by 2
31 more
  1. 2025Hashing-Baseline: Rethinking Hashing in the Age of Pretrained Modelscited by 1
  2. 2025SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filteringcited by 1
  3. 2026UniWhisper: Efficient Continual Multi-task Training for Robust Universal Audio Representationcited by 1
  4. 2026Unlocking Strong Supervision: A Data-Centric Study of General-Purpose Audio Pre-Training Methodscited by 1
  5. 2026ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Modelscited by 1
  6. 2026VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentationcited by 1
  7. 2026Can We Predict Anomaly Detection Performance from Embedding-Space Geometry?cited by 1
  8. 2024Multimodal Laryngoscopic Video Analysis for Assisted Diagnosis of Vocal Fold Paralysiscited by 0
  9. 2025Revisiting SSL for sound event detection: complementary fusion and adaptive post-processingcited by 0
  10. 2025AudioMAE++: learning better masked audio representations with SwiGLU FFNscited by 0
  11. 2025ECHO: Frequency-aware Hierarchical Encoding for Variable-length Signalscited by 0
  12. 2025An overview of neural architectures for self-supervised audio representation learning from masked spectrogramscited by 0
  13. 2025Unify Variables in Neural Scaling Laws for General Audio Representations via Embedding Effective Rankcited by 0
  14. 2025Better audio representations are more brain-like: linking model-brain alignment with performance in downstream auditory taskscited by 0
  15. 2026USAD 2.0: Scaling Representation Distillation for Universal Audio Understandingcited by 0
  16. 2026LambdaMark: Semantic Audio Watermarking for Robustness and Radioactivitycited by 0
  17. 2026MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learningcited by 0
  18. 2026BEST-RQ-2: Contextualize-Then-Predict, a Two-Step Approach for Self-Supervised Audio Representationscited by 0
  19. 2026Large Audio Language Models for Spoofing-Aware Speaker Verificationcited by 0
  20. 2026Alignment Is All You Need: Instruction-Free Training for General Audio-Language Modelscited by 0
  21. 2026Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learnerscited by 0
  22. 2026Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusioncited by 0
  23. 2026MultiHuSE: A Multimodal Dataset for Humour Styles and Emotionscited by 0
  24. 2026Correlation-Guided Encoder Selection for Multi-Encoder Large Audio-Language Modelscited by 0
  25. 2026VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrievalcited by 0
  26. 2026Bearings: Self-Supervised Soundfield Embeddings from First-Order Ambisonicscited by 0
  27. 2026Modality-Gated Deep Adapters: Adding a Modality to a Frozen Embedding Model with Exact Preservationcited by 0
  28. 2026Anomaly-Free Self-Optimization via AUC Boundscited by 0
  29. 2026VoiceNet: Fine-Grained Voice Understanding Beyond Emotion at Scalecited by 0
  30. 2026UniAE-MoE: A Unified Audio Encoder via Mixture of Expertscited by 0
  31. 2026Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillationcited by 0