mlsanityData to 7 Oct 2026

A Human Subject Study of Named Entity Recognition (NER) in Conversational Music Recommendation Queries

arXiv 2303.06944 ↗First posted 13 Mar 2023cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We conducted a human subject study of named entity recognition on a noisy corpus of conversational music recommendation queries, with many irregular and novel named entities. We evaluated the human NER linguistic behaviour in these challenging conditions and compared it with the most common NER systems nowadays, fine-tuned transformers. Our goal was to learn about the task to guide the design of better evaluation methods and NER algorithms. The results showed that NER in our context was quite hard for both human and algorithms under a strict evaluation schema; humans had higher precision, while the model higher recall because of entity exposure especially during pre-training; and entity types had different error patterns (e.g. frequent typing errors for artists). The released corpus goes beyond predefined frames of interaction and can support future work in conversational music recommendation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 58 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

24 of 58 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2020Revisiting Few-sample BERT Fine-tuningcited by 237
  8. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 150
  9. 2020A Survey on Conversational Recommender Systemscited by 142
  10. 2019A Survey on Recent Advances in Named Entity Recognition from Deep Learning modelscited by 117
  11. 2018Learning Named Entity Tagger using Domain-Specific Dictionarycited by 65
  12. 2020Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsingcited by 59
12 more
  1. 2020Named Entity Recognition without Labelled Data: A Weak Supervision Approachcited by 46
  2. 2017Current Challenges and Visions in Music Recommender Systems Researchcited by 42
  3. 2020Rethinking Generalization of Neural Models: A Named Entity Recognition Case Studycited by 35
  4. 2019Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmarkcited by 32
  5. 2021Memorisation versus Generalisation in Pre-trained Language Modelscited by 32
  6. 2020TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognitioncited by 30
  7. 2020Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019cited by 20
  8. 2022Flow Moods: Recommending Music by Moods on Deezercited by 11
  9. 2021Probing Pre-trained Auto-regressive Language Models for Named Entity Typing and Recognitioncited by 10
  10. 2020An End-to-End Solution for Named Entity Recognition in eCommerce Searchcited by 6
  11. 2020Noise Robust Named Entity Understanding for Voice Assistantscited by 5
  12. 2019Recognizing Musical Entities in User-generated Contentcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 4.

6 papers.

  1. 2024A Benchmark and Robustness Study of In-Context-Learning with Large Language Models in Music Entity Detectioncited by 4
  2. 2025Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluationcited by 1
  3. 2026HyCoRec: Hypergraph-Enhanced Multi-Preference Learning for Alleviating Matthew Effect in Conversational Recommendationcited by 1
  4. 2026Mitigating Matthew Effect: Multi-Hypergraph Boosted Multi-Interest Self-Supervised Learning for Conversational Recommendationcited by 1
  5. 2026Beyond Musical Descriptors: Extracting Preference-Bearing Intent in Music Queriescited by 0
  6. 2026FacetCRS: Multi-Faceted Preference Learning for Pricking Filter Bubbles in Conversational Recommender Systemcited by 0