# Hearing's Blind Spots: Toward Perceptual Compute Allocation in Neural Text-to-Speech

arXiv: https://arxiv.org/abs/2610.14056 · HTML: https://mlsanity.com/paper/2610.14056/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 21 of 21)

- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [Mixture-of-Depths: Dynamically allocating compute in transformer-based language models](https://mlsanity.com/paper/2404.02258/) (2024)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2203.02395/) (2022)
- [Psychoacoustic Calibration of Loss Functions for Efficient End-to-End Neural Audio Coding](https://mlsanity.com/paper/2101.00054/) (2020)
- [Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decoding](https://mlsanity.com/paper/2505.15380/) (2025)
- [Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling](https://mlsanity.com/paper/2505.19931/) (2025)
- [Multi-band Frequency Reconstruction for Neural Psychoacoustic Coding](https://mlsanity.com/paper/2505.07235/) (2025)
- [DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Models](https://mlsanity.com/paper/2406.05464/) (2024)
- [Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policy](https://mlsanity.com/paper/2506.22023/) (2025)
- [Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generation](https://mlsanity.com/paper/2603.23491/) (2026)
- [On Psychoacoustically Weighted Cost Functions Towards Resource-Efficient Deep Neural Networks for Speech Denoising](https://mlsanity.com/paper/1801.09774/) (2018)
- [SPAR-K: Scheduled Periodic Alternating Early Exit for Spoken Language Models](https://mlsanity.com/paper/2603.09215/) (2026)

## Abstract (arXiv)

A speech synthesizer produces a detailed waveform, but a listener need not hear every detail. Strong sounds can mask weaker neighbors, creating context-dependent "blind spots" in hearing. Could neural text-to-speech (TTS) spend less computation on those details? We investigate a first step: how the placement of spectral degradation affects completed speech from Piper and Kokoro. At matched time-frequency bin-removal budgets, a simple low-energy-first policy achieves higher mean wideband PESQ than random deletion and the tested psychoacoustic masking criteria. At 50% removal in the spreading-matrix comparison, energy, masking, and random policies score 4.46, 3.60, and 1.19 on Piper, and 3.42, 2.10, and 1.22 on Kokoro. Spectrograms and masking maps expose where these perturbations fall and why a model's below-threshold regions cannot automatically be treated as inaudible. Calibration diagnostics show substantial threshold sensitivity, while a small single-listener pilot establishes no transparent removal budget. CPU profiles identify waveform decoding as the main measured kernel cost, but a tested dynamic-INT8 Kokoro decoder runs slower than FP32 on both CPUs. The results establish energy ranking as a strong oracle baseline and motivate testing perceptual allocation inside a decoder. They do not yet demonstrate saved computation: the decisions here use already synthesized audio, and quality metrics are evaluated at 16 kHz. We release scripts, results, audio examples, and a listening-study instrument.