mlsanityData to 7 Oct 2026

Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home

arXiv 2306.00482 ↗First posted 1 Jun 2023cs.CY cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data with kids going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction tasks in the math learning domain. To recognize kids' speech in realistic home environments, we investigate several ASR systems, including the commercial Google Cloud and the latest open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children for math learning in authentic homes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 45 of its 118 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

45 of 118 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  12. 2021GPT Understands, Toocited by 715
33 more
  1. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  2. 2023Mathematical Capabilities of ChatGPTcited by 263
  3. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  4. 2019Benchmarking Natural Language Understanding Services for building Conversational Agentscited by 166
  5. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
  6. 2019ConveRT: Efficient and Accurate Conversational Representations from Transformerscited by 95
  7. 2020DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialoguecited by 89
  8. 2022The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialoguescited by 78
  9. 2017Rasa: Open Source Language Understanding and Dialogue Managementcited by 67
  10. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58
  11. 2020Language Models as Few-Shot Learner for Task-Oriented Dialogue Systemscited by 41
  12. 2022The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Movescited by 38
  13. 2023Transformers in Speech Processing: A Surveycited by 38
  14. 2021MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Educationcited by 37
  15. 2022StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvementcited by 37
  16. 2020DIET: Lightweight Language Understanding for Dialogue Systemscited by 35
  17. 2015A Survey of Available Corpora for Building Data-Driven Dialogue Systemscited by 33
  18. 2023Opportunities and Challenges in Neural Dialog Tutoringcited by 33
  19. 2017StarSpace: Embed All The Things!cited by 24
  20. 2022LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learningcited by 24
  21. 2018Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendationscited by 21
  22. 2022Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue Systemcited by 16
  23. 2019Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLUcited by 13
  24. 2021N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypothesescited by 12
  25. 2021Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognitioncited by 9
  26. 2019Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterancescited by 8
  27. 2019Modeling Intent, Dialog Policies and Response Adaptation for Goal-Oriented Interactionscited by 5
  28. 2021Semi-supervised Interactive Intent Labelingcited by 5
  29. 2021Improving Controllability of Educational Question Generation by Keyword Provisioncited by 3
  30. 2020Improving Classification through Weak Supervision in Context-specific Conversational Agent Development for Teacher Educationcited by 2
  31. 2022NLU for Game-based Learning in Real: Initial Evaluationscited by 2
  32. 2022End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematicscited by 2
  33. 2023Gamifying Math Education using Object Detectioncited by 2

Built on by

Not cited yet.