# SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metrics

arXiv: https://arxiv.org/abs/2401.16812 · HTML: https://mlsanity.com/paper/2401.16812/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2024.

## Built on by (most cited first, 30 of 38)

- [Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey](https://mlsanity.com/paper/2409.11564/) (2024)
- [AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement](https://mlsanity.com/paper/2501.15417/) (2025)
- [LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement](https://mlsanity.com/paper/2503.00493/) (2025)
- [Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation](https://mlsanity.com/paper/2409.16644/) (2024)
- [Universal Speech Enhancement with Regression and Generative Mamba](https://mlsanity.com/paper/2505.21198/) (2025)
- [Multi-Metric Preference Alignment for Generative Speech Restoration](https://mlsanity.com/paper/2508.17229/) (2025)
- [LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Models](https://mlsanity.com/paper/2504.07402/) (2025)
- [PASE: Leveraging the Phonological Prior of WavLM for Low-Hallucination Generative Speech Enhancement](https://mlsanity.com/paper/2511.13300/) (2025)
- [FlowSE-GRPO: Training Flow Matching Speech Enhancement via Online Reinforcement Learning](https://mlsanity.com/paper/2601.16483/) (2026)
- [Articulatory Feature Prediction from Surface EMG during Speech Production](https://mlsanity.com/paper/2505.13814/) (2025)
- [Voice of a Continent: Mapping Africa's Speech Technology Frontier](https://mlsanity.com/paper/2505.18436/) (2025)
- [Real-Time Streamable Generative Speech Restoration with Flow Matching](https://mlsanity.com/paper/2512.19442/) (2025)
- [SenSE: Semantic-Aware High-Fidelity Universal Speech Enhancement](https://mlsanity.com/paper/2509.24708/) (2025)
- [Schr"odinger Bridge Mamba for One-Step Speech Enhancement](https://mlsanity.com/paper/2510.16834/) (2025)
- [TS-URGENet: A Three-stage Universal Robust and Generalizable Speech Enhancement Network](https://mlsanity.com/paper/2505.18533/) (2025)
- [High-Fidelity Speech Enhancement via Discrete Audio Tokens](https://mlsanity.com/paper/2510.02187/) (2025)
- [TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosis](https://mlsanity.com/paper/2604.22225/) (2026)
- [Incorporating Linguistic Constraints from External Knowledge Source for Audio-Visual Target Speech Extraction](https://mlsanity.com/paper/2506.09792/) (2025)
- [GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model](https://mlsanity.com/paper/2512.20978/) (2025)
- [Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Tasks](https://mlsanity.com/paper/2601.12205/) (2026)
- [Geneses: Unified Generative Speech Enhancement and Separation](https://mlsanity.com/paper/2601.18456/) (2026)
- [Residual Tokens Enhance Masked Autoencoders for Speech Modeling](https://mlsanity.com/paper/2601.19399/) (2026)
- [StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model](https://mlsanity.com/paper/2604.19635/) (2026)
- [BnTTS: Few-Shot Speaker Adaptation in Low-Resource Setting](https://mlsanity.com/paper/2502.05729/) (2025)
- [Score-Based Training for Energy-Based TTS Models](https://mlsanity.com/paper/2505.13771/) (2025)
- [SpeechOp: Inference-Time Task Composition for Generative Speech Processing](https://mlsanity.com/paper/2509.14298/) (2025)
- [Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions](https://mlsanity.com/paper/2509.24457/) (2025)
- [ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extraction](https://mlsanity.com/paper/2511.06288/) (2025)
- [SLM-SS: Speech Language Model for Generative Speech Separation](https://mlsanity.com/paper/2601.19533/) (2026)
- [SLD-L2S: Hierarchical Subspace Latent Diffusion for High-Fidelity Lip to Speech Synthesis](https://mlsanity.com/paper/2602.11477/) (2026)

## Abstract (arXiv)

While subjective assessments have been the gold standard for evaluating speech generation, there is a growing need for objective metrics that are highly correlated with human subjective judgments due to their cost efficiency. This paper proposes reference-aware automatic evaluation methods for speech generation inspired by evaluation metrics in natural language processing. The proposed SpeechBERTScore computes the BERTScore for self-supervised dense speech features of the generated and reference speech, which can have different sequential lengths. We also propose SpeechBLEU and SpeechTokenDistance, which are computed on speech discrete tokens. The evaluations on synthesized speech show that our method correlates better with human subjective ratings than mel cepstral distortion and a recent mean opinion score prediction model. Also, they are effective in noisy speech evaluation and have cross-lingual applicability.