# LASER: An LLM-based ASR Scoring and Evaluation Rubric

arXiv: https://arxiv.org/abs/2510.07437 · HTML: https://mlsanity.com/paper/2510.07437/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding](https://mlsanity.com/paper/2104.02138/) (2021)
- [IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages](https://mlsanity.com/paper/2403.01926/) (2024)
- [Transformer based Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2004.06338/) (2020)
- [Aligning ASR Evaluation with Human and LLM Judgments: Intelligibility Metrics Using Phonetic, Semantic, and NLI Approaches](https://mlsanity.com/paper/2506.16528/) (2025)
- [Personalized Predictive ASR for Latency Reduction in Voice Assistants](https://mlsanity.com/paper/2305.13794/) (2023)
- [Neural Compound-Word (Sandhi) Generation and Splitting in Sanskrit Language](https://mlsanity.com/paper/2010.12940/) (2020)

## Built on by (most cited first, 5 of 5)

- [Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation](https://mlsanity.com/paper/2605.29430/) (2026)
- [Interactive ASR: Towards Human-Like Interaction and Semantic Coherence Evaluation for Agentic Speech Recognition](https://mlsanity.com/paper/2604.09121/) (2026)
- [Designing large language model prompts to extract scores from messy text: A shared dataset and challenge](https://mlsanity.com/paper/2601.18271/) (2026)
- [Context-Aware Interleaved Batching for WhisperX](https://mlsanity.com/paper/2608.31170/) (2026)
- [mu-bench: A Multilingual Utterance Transcription Benchmark](https://mlsanity.com/paper/2609.32082/) (2026)

## Abstract (arXiv)

Standard ASR evaluation metrics like Word Error Rate (WER) tend to unfairly penalize morphological and syntactic nuances that do not significantly alter sentence semantics. We introduce an LLM-based scoring rubric LASER that leverages state-of-the-art LLMs' in-context learning abilities to learn from prompts with detailed examples. Hindi LASER scores using Gemini 2.5 Pro achieved a very high correlation score of 94% with human annotations. Hindi examples in the prompt were also effective in analyzing errors in other Indian languages such as Marathi, Kannada and Malayalam. We also demonstrate how a smaller LLM like Llama 3 can be finetuned on word-pair examples derived from reference and ASR predictions to predict what kind of penalty should be applied with close to 89% accuracy.