mlsanityData to 14 Oct 2026

Ruby-ASR: Evidence-Preserving Supervision for Joint Orthographic and Lexical-Reading Recognition

arXiv 2609.27289 ↗First posted 23 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Conventional Japanese automatic speech recognition (ASR) is supervised by an orthographic transcript, although the same written form can correspond to different lexical readings realized in speech. Such utterances receive an identical target, so their reading distinction is absent from the supervision interface and cannot be recovered reliably by post-hoc text-only grapheme-to-phoneme conversion. We present Ruby-ASR, which refines the conventional target into a span-bound orthographic--lexical-reading sequence. Unlike separate full-sentence orthographic and phonological outputs, the ruby representation locally binds each written span to its realized reading and permits deterministic recovery of both views. We instantiate the target under subtitle-style and verbatim-style transcription conventions using a Qwen3-ASR backbone; a mora-level CTC objective provides auxiliary monotonic reading supervision. The experimental results across five Japanese benchmarks show that refining the recognition target can improve lexical-reading recovery without sacrificing readable orthographic transcription. We release the checkpoints and inference code.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

18 of 30 references.

References · 18

  1. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  2. 2026Qwen3-ASR Technical Reportcited by 118
  3. 2023Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labellingcited by 90
  4. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 58
  5. 2023Lenient Evaluation of Japanese Speech Recognition: Modeling Naturally Occurring Spelling Inconsistencycited by 10
  6. 2021Construction of a Large-scale Japanese ASR Corpus on TV Recordingscited by 8
  7. 2024Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognitioncited by 7
  8. 2024Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decodingcited by 7
  9. 2021A Comparative Study on Neural Architectures and Training Methods for Japanese Speech Recognitioncited by 6
  10. 2023Retraining-free Customized ASR for Enharmonic Words Based on a Named-Entity-Aware Model and Phoneme Similarity Estimationcited by 5
  11. 2025Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognitioncited by 5
  12. 2026Benchmarking Large Language Models for Grapheme-to-Phoneme Conversion: A Japanese Case Studycited by 3
6 more
  1. 2026Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesiscited by 3
  2. 2021A Dual-Decoder Conformer for Multilingual Speech Recognitioncited by 1
  3. 2025Transcript-Prompted Whisper with Dictionary-Enhanced Decoding for Japanese Speech Annotationcited by 1
  4. 2026Distilling LLM Semantic Priors into Encoder-Only Multi-Talker ASR with Talker-Count Routingcited by 1
  5. 2026Beyond Acoustic Prefixes: Persistent Grounding in Serialized Acoustic Memory for LLM-Based Multi-Talker Speech Recognitioncited by 1
  6. 2026YOMI-Bench: A Benchmark for Evaluating Kanji Reading and Phonological Understanding of LLMs for Japanesecited by 1

Built on by

Not cited yet.