# Back to Ear: Perceptually Driven High Fidelity Music Reconstruction

arXiv: https://arxiv.org/abs/2509.14912 · HTML: https://mlsanity.com/paper/2509.14912/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 13 of its 17 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 13 of 13)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [FMA: A Dataset For Music Analysis](https://mlsanity.com/paper/1612.01840/) (2016)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)
- [MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music](https://mlsanity.com/paper/2402.09871/) (2024)
- [DISCO-10M: A Large-Scale Music Dataset](https://mlsanity.com/paper/2306.13512/) (2023)

## Built on by (most cited first, 4 of 4)

- [Stable Audio 3](https://mlsanity.com/paper/2605.17991/) (2026)
- [SAME: A Semantically-Aligned Music Autoencoder](https://mlsanity.com/paper/2605.18613/) (2026)
- [STAR-VAE: Structured Topology-Aware Regularization for Audio Reconstruction and Generation](https://mlsanity.com/paper/2606.23064/) (2026)
- [Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding](https://mlsanity.com/paper/2606.27320/) (2026)

## Abstract (arXiv)

Variational Autoencoders (VAEs) are essential for large-scale audio tasks like diffusion-based generation. However, existing open-source models often neglect auditory perceptual aspects during training, leading to weaknesses in phase accuracy and stereophonic spatial representation. To address these challenges, we propose εar-VAE, an open-source music signal reconstruction model that rethinks and optimizes the VAE training paradigm. Our contributions are threefold: (i) A K-weighting perceptual filter applied prior to loss calculation to align the objective with auditory perception. (ii) Two novel phase losses: a Correlation Loss for stereo coherence, and a Phase Loss using its derivatives--Instantaneous Frequency and Group Delay--for precision. (iii) A new spectral supervision paradigm where magnitude is supervised by all four Mid/Side/Left/Right components, while phase is supervised only by the LR components. Experiments show εar-VAE at 44.1kHz substantially outperforms leading open-source models across diverse metrics, showing particular strength in reconstructing high-frequency harmonics and the spatial characteristics.