# AgenticASR: Refining Speech Recognition in Real-World Scenarios via an Agentic Approach

arXiv: https://arxiv.org/abs/2607.28175 · HTML: https://mlsanity.com/paper/2607.28175/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [Gemma 4 Technical Report](https://mlsanity.com/paper/2607.02770/) (2026)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Qwen3-ASR Technical Report](https://mlsanity.com/paper/2601.21337/) (2026)
- [Describe Anything: Detailed Localized Image and Video Captioning](https://mlsanity.com/paper/2504.16072/) (2025)
- [MiniCPM4: Ultra-Efficient LLMs on End Devices](https://mlsanity.com/paper/2506.07900/) (2025)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Improving Readability for Automatic Speech Recognition Transcription](https://mlsanity.com/paper/2004.04438/) (2020)
- [Improving Punctuation Restoration for Speech Transcripts via External Data](https://mlsanity.com/paper/2110.00560/) (2021)
- [MMAE: A Massive Multitask Audio Editing Benchmark](https://mlsanity.com/paper/2606.07229/) (2026)
- [Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model](https://mlsanity.com/paper/2202.08171/) (2022)
- [From Disfluency Detection to Intent Detection and Slot Filling](https://mlsanity.com/paper/2209.08359/) (2022)
- [Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts](https://mlsanity.com/paper/2408.09688/) (2024)
- [Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation](https://mlsanity.com/paper/2605.29430/) (2026)
- [Interactive ASR: Towards Human-Like Interaction and Semantic Coherence Evaluation for Agentic Speech Recognition](https://mlsanity.com/paper/2604.09121/) (2026)
- [FormalASR: End-to-End Spoken Chinese to Formal Text](https://mlsanity.com/paper/2605.19266/) (2026)

## Built on by (most cited first, 1 of 1)

- [Talk2Agent: Benchmarking Voice Interfaces for Text Agents](https://mlsanity.com/paper/2609.38867/) (2026)

## Abstract (arXiv)

Automatic speech recognition (ASR) has achieved substantial gains in transcription accuracy, yet verbatim transcription does not necessarily produce readily usable text. It retains fillers, repetitions, false starts, and self-corrections that increase reading effort, obscure the speaker's final intent, and propagate unresolved or abandoned content to downstream tasks. Existing spoken-to-written methods process completed audio or transcripts but cannot revise emitted text when later speech changes how preceding content should be interpreted. We therefore formulate Agentic Speech Recognition (AgenticSR), an audio-to-clean-text task that removes disfluencies, resolves self-corrections, and normalizes written form while preserving the speaker's final intent. AgenticASR implements this task through an ASR--Refiner architecture that repeatedly transforms a bounded active context and replaces its corresponding output span as audio arrives. This enables continual emission and revision over streams of arbitrary duration. We also introduce AASR-Bench, a bilingual benchmark with fine-grained atomic rubrics. Across multiple ASR front ends, AgenticASR attains the highest AASR-Bench scores among evaluated systems. A human--AI agreement study shows that rubric-based judgments align with independent expert assessments. Ablations characterize Refiner capacity, context length, and the quality--latency trade-off between online and offline inference. Together, these results establish AgenticASR as a practical framework for intent-preserving clean transcription during ongoing speech. Code, AASR-Bench, and a demo will be released at https://github.com/AnXMuy/AgenticASR.