mlsanityData to 7 Oct 2026

Play It Back: Iterative Attention for Audio Recognition

arXiv 2210.11328 ↗First posted 20 Oct 2022cs.SD cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine-grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the-art performance across three audio-classification benchmarks: AudioSet, VGG-Sound, and EPIC-KITCHENS-100.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 37 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

29 of 37 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  9. 2020Object-Centric Learning with Slot Attentioncited by 865
  10. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
  11. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  12. 2020Rescaling Egocentric Visioncited by 529
17 more
  1. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  2. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  3. 2022Masked Autoencoders that Listencited by 271
  4. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  5. 2021Efficient Training of Audio Transformers with Patchoutcited by 222
  6. 2022HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detectioncited by 174
  7. 2022MAE-AST: Masked Autoencoding Audio Spectrogram Transformercited by 82
  8. 2021PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregationcited by 81
  9. 2021PolyViT: Co-training Vision Transformers on Images, Videos and Audiocited by 58
  10. 2018Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networkscited by 56
  11. 2021Slow-Fast Auditory Streams For Audio Recognitioncited by 56
  12. 2022Masked Spectrogram Prediction For Self-Supervised Audio Pre-Trainingcited by 50
  13. 2018A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labelingcited by 37
  14. 2019Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimizationcited by 29
  15. 2016Visual Concept Recognition and Localization via Iterative Introspectioncited by 6
  16. 2016AutoScaler: Scale-Attention Networks for Visual Correspondencecited by 5
  17. 2022Learning Temporal Resolution in Spectrogram for Audio Classificationcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2.

3 papers.

  1. 2022Audiovisual Masked Autoencoderscited by 47
  2. 2023Multimodal Distillation for Egocentric Action Recognitioncited by 28
  3. 2023Zorro: the masked multimodal transformercited by 13