mlsanityData to 7 Oct 2026

Automatic Generation of Behavioral Test Cases For Natural Language Processing Using Clustering and Prompting

arXiv 2408.00161 ↗First posted 31 Jul 2024cs.CL cs.AI cs.ETcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent work in behavioral testing for natural language processing (NLP) models, such as Checklist, is inspired by related paradigms in software engineering testing. They allow evaluation of general linguistic capabilities and domain understanding, hence can help evaluate conceptual soundness and identify model weaknesses. However, a major challenge is the creation of test cases. The current packages rely on semi-automated approach using manual development which requires domain expertise and can be time consuming. This paper introduces an automated approach to develop test cases by exploiting the power of large language models and statistical techniques. It clusters the text representations to carefully construct meaningful groups and then apply prompting techniques to automatically generate Minimal Functionality Tests (MFT). The well-known Amazon Reviews corpus is used to demonstrate our approach. We analyze the behavioral test profiles across four different classification algorithms and discuss the limitations and strengths of those models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 21 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

11 of 21 references.

References · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  9. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  10. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  11. 2020The Multilingual Amazon Reviews Corpuscited by 142

Built on by

Not cited yet.