Unpacking Robustness in Inflectional Languages: Adversarial Evaluation and Mechanistic Insights
On the map: Mechanistic interpretability → Steering and probing for LLM safety
Abstract (arXiv)
Various techniques are used in the generation of adversarial examples, including methods such as TextBugger which introduce minor, hardly visible perturbations to words leading to changes in model behaviour. Another class of techniques involves substituting words with their synonyms in a way that preserves the text's meaning but alters its predicted class, with TextFooler being a prominent example of such attacks. Most adversarial example generation methods are developed and evaluated primarily on non-inflectional languages, typically English. In this work, we evaluate and explain how adversarial attacks perform in inflectional languages. To explain the impact of inflection on model behaviour and its robustness under attack, we designed a novel protocol inspired by mechanistic interpretability, based on Edge Attribution Patching (EAP) method. The proposed evaluation protocol relies on parallel task-specific corpora that include both inflected and syncretic variants of texts in two languages -- Polish and English. To analyse the models and explain the relationship between inflection and adversarial robustness, we create a new benchmark based on task-oriented dataset MultiEmo, enabling the identification of mechanistic inflection-related elements of circuits within the model and analyse their behaviour under attack.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability. Builds on 17 of its 32 references that are in the map. First posted on arXiv 8 May 2025.
Builds on
17 of 32 references.
References · 17
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2013Intriguing properties of neural networkscited by 5,155
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
- 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
- 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
5 more
- 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 149
- 2021HerBERT: Efficiently Pretrained Transformer-based Language Model for Polishcited by 51
- 2021RobeCzech: Czech RoBERTa, a monolingual contextualized language representation modelcited by 33
- 2017Czech Text Document Corpus v 2.0cited by 4
- 2024Selecting Between BERT and GPT for Text Classification in Political Science Researchcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarks
- 2024Adversarial Attacks on Parts of Speech: An Empirical Study in Text-to-Image Generation
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
- 2022Rethinking Textual Adversarial Defense for Pre-trained Language Models
- 2024A Constraint-Enforcing Reward for Adversarial Attacks on Text Classifiers
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models
- 2025Toward Understanding the Transferability of Adversarial Suffixes in Large Language Models
- 2026Unifying Adversarial Robustness and Training Across Text Scoring Models