# The Eloquence submission for Task 2 of the Interspeech 2026 MLC-SLM challenge

arXiv: https://arxiv.org/abs/2609.11724 · HTML: https://mlsanity.com/paper/2609.11724/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 28 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Builds on (most cited first, 24 of 24)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.04779/) (2025)
- [A Survey on Speech Large Language Models for Understanding](https://mlsanity.com/paper/2410.18908/) (2024)
- [SLM: Bridge the thin gap between speech and text foundation models](https://mlsanity.com/paper/2310.00230/) (2023)
- [TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context](https://mlsanity.com/paper/2110.04410/) (2021)
- [Voxtral](https://mlsanity.com/paper/2507.13264/) (2025)
- [MSU-Bench: Towards Understanding the Conversational Multi-talker Scenarios](https://mlsanity.com/paper/2508.08155/) (2025)
- [Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods](https://mlsanity.com/paper/2509.13785/) (2025)
- [Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio](https://mlsanity.com/paper/2511.16046/) (2025)
- [Closing the Modality Reasoning Gap for Speech Large Language Models](https://mlsanity.com/paper/2601.05543/) (2026)
- [Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languages](https://mlsanity.com/paper/2508.05149/) (2025)
- [AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks](https://mlsanity.com/paper/2506.23049/) (2025)
- [The Eloquence team submission for task 1 of MLC-SLM challenge](https://mlsanity.com/paper/2507.19308/) (2025)
- [Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models](https://mlsanity.com/paper/2508.07273/) (2025)
- [Benchmarking Humans and Machines on Complex Multilingual Speech Understanding Tasks](https://mlsanity.com/paper/2509.17965/) (2025)
- [A Unified and Reproducible Experimentation Framework for Speech Understanding](https://mlsanity.com/paper/2605.30899/) (2026)
- [Balancing ASR and diarization in end-to-end LLMs for multi-talker speech recognition](https://mlsanity.com/paper/2606.13095/) (2026)

## Abstract (arXiv)

This paper details the Eloquence team's approach to Task 2 of the 2nd MLC-SLM challenge at Interspeech 2026, which involves multilingual Multiple-Choice Question Answering (MCQA) across 21 languages. Three approaches are explored. First, we fine-tune Voxtral-Mini-3B via LoRA with cross-lingual data augmentation, ASR transcript augmentation and timestamp-aware audio cropping, achieving 0.72 macro-accuracy on evaluation Phase 2. Second, we apply multimodal in-context learning (ICL) to the frozen Voxtral-24B model to correct a strong label bias, reaching 0.81, our best result. Third, a training-free retrieval system based on a three-layer voice-anchored memory combining acoustic identity, semantic content, and a knowledge graph achieves 0.68. All three systems substantially outperform the official baseline.