# Session-Level Spoken Language Assessment with a Multimodal Foundation Model via Multi-Target Learning

arXiv: https://arxiv.org/abs/2509.16025 · HTML: https://mlsanity.com/paper/2509.16025/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 8 of 8)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [Assessment of L2 Oral Proficiency using Speech Large Language Models](https://mlsanity.com/paper/2505.21148/) (2025)
- [The NTNU System at the S&I Challenge 2025 SLA Open Track](https://mlsanity.com/paper/2506.05121/) (2025)
- [Acoustically Precise Hesitation Tagging Is Essential for End-to-End Verbatim Transcription Systems](https://mlsanity.com/paper/2506.04076/) (2025)

## Built on by (most cited first, 2 of 2)

- [LOPA: Enhancing Spoken Language Assessment via Latent Ordinal Prototype Alignment](https://mlsanity.com/paper/2606.31310/) (2026)
- [CASA: Content-Acoustic Speaking Assessment with Speech Encoder and Large Language Model](https://mlsanity.com/paper/2608.13101/) (2026)

## Abstract (arXiv)

Spoken Language Assessment (SLA) estimates a learner's oral proficiency from spontaneous speech. The growing population of L2 English speakers has intensified the demand for reliable SLA, a critical component of Computer Assisted Language Learning (CALL). Existing efforts often rely on cascaded pipelines, which are prone to error propagation, or end-to-end models that often operate on a short audio window, which might miss discourse-level evidence. This paper introduces a novel multimodal foundation model approach that performs session-level evaluation in a single pass. Our approach couples multi-target learning with a frozen, Whisper ASR model-based speech prior for acoustic-aware calibration, allowing for jointly learning holistic and trait-level objectives of SLA without resorting to handcrafted features. By coherently processing the entire response session of an L2 speaker, the model excels at predicting holistic oral proficiency. Experiments conducted on the Speak & Improve benchmark demonstrate that our proposed approach outperforms the previous state-of-the-art cascaded system and exhibits robust cross-part generalization, producing a compact deployable grader that is tailored for CALL applications.