# Enhancing Robustness in Biomedical NLI Models: A Probing Approach for Clinical Trials

arXiv: https://arxiv.org/abs/2402.02558 · HTML: https://mlsanity.com/paper/2402.02558/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 13 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Probing Natural Language Inference Models through Semantic Fragments](https://mlsanity.com/paper/1909.07521/) (2019)
- [Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2009.07964/) (2020)
- [RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoning](https://mlsanity.com/paper/2205.12598/) (2022)
- [Sebis at SemEval-2023 Task 7: A Joint System for Natural Language Inference and Evidence Retrieval from Clinical Trial Reports](https://mlsanity.com/paper/2304.13180/) (2023)
- [Does My Representation Capture X? Probe-Ably](https://mlsanity.com/paper/2104.05807/) (2021)
- [Interventional Probing in High Dimensions: An NLI Case Study](https://mlsanity.com/paper/2304.10346/) (2023)
- [THiFLY Research at SemEval-2023 Task 7: A Multi-granularity System for CTR-based Textual Entailment and Evidence Retrieval](https://mlsanity.com/paper/2306.01245/) (2023)
- [Estimating the Causal Effects of Natural Logic Features in Neural NLI Models](https://mlsanity.com/paper/2305.08572/) (2023)

## Abstract (arXiv)

Large Language Models have revolutionized various fields and industries, such as Conversational AI, Content Generation, Information Retrieval, Business Intelligence, and Medical, to name a few. One major application in the field of medical is to analyze and investigate clinical trials for entailment tasks.However, It has been observed that Large Language Models are susceptible to shortcut learning, factual inconsistency, and performance degradation with little variation in context. Adversarial and robust testing is performed to ensure the integrity of models output. But, ambiguity still persists. In order to ensure the integrity of the reasoning performed and investigate the model has correct syntactic and semantic understanding probing is used. Here, I used mnestic probing to investigate the Sci-five model, trained on clinical trial. I investigated the model for feature learnt with respect to natural logic. To achieve the target, I trained task specific probes. Used these probes to investigate the final layers of trained model. Then, fine tuned the trained model using iterative null projection. The results shows that model accuracy improved. During experimentation, I observed that size of the probe has affect on the fine tuning process.