Spiking Music: Audio Compression with Event Based Auto-encoders
Not on the map. Why?
Abstract (arXiv)
Neurons in the brain communicate information via punctual events called spikes. The timing of spikes is thought to carry rich information, but it is not clear how to leverage this in digital systems. We demonstrate that event-based encoding is efficient for audio compression. To build this event-based representation we use a deep binary auto-encoder, and under high sparsity pressure, the model enters a regime where the binary event matrix is stored more efficiently with sparse matrix storage algorithms. We test this on the large MAESTRO dataset of piano recordings against vector quantized auto-encoders. Not only does our "Spiking Music compression" algorithm achieve a competitive compression/reconstruction trade-off, but selectivity and synchrony between encoded events and piano key strikes emerge without supervision in the sparse regime.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 55 references that are in the map. First posted on arXiv 2 Feb 2024.
Builds on
34 of 55 references.
References · 34
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2017Neural Discrete Representation Learningcited by 4,650
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2016End-to-end Optimized Image Compressioncited by 673
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
- 2020Jukebox: A Generative Model for Musiccited by 535
- 2023Simple and Controllable Music Generationcited by 480
- 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
22 more
- 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
- 2023Language Modeling Is Compressioncited by 225
- 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
- 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 179
- 2019The Heidelberg spiking datasets for the systematic evaluation of spiking neural networkscited by 135
- 2016Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computingcited by 68
- 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
- 2020Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networkscited by 59
- 2023A Unified, Scalable Framework for Neural Population Decodingcited by 59
- 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 47
- 2020SEANet: A Multi-modal Speech Enhancement Networkcited by 35
- 2018Self-Supervised Video Hashing with Hierarchical Binary Auto-encodercited by 34
- 2022Efficient recurrent architectures through activity sparsity and sparse back-propagation through timecited by 26
- 2021Variable-rate discrete representation learningcited by 24
- 2023High-performance deep spiking neural networks with 0.3 spikes per neuroncited by 22
- 2018Fitting New Speakers Based on a Short Untranscribed Samplecited by 18
- 2020Single channel voice separation for unknown number of speakers under reverberant and noisy settingscited by 10
- 2023The Intel Neuromorphic DNS Challengecited by 9
- 2019An efficient and perceptually motivated auditory neural encoding and decoding algorithm for spiking neural networkscited by 8
- 2021Fitting summary statistics of neural data with a differentiable spiking network simulatorcited by 5
- 2021A Differentiable Point Process with Its Application to Spiking Neural Networkscited by 4
- 2023Trial matching: capturing variability with data-constrained spiking neural networkscited by 4
Built on by
Citing papers per year: 2024: 1.
1 paper.