# Reproducibility, Replicability and Beyond: Assessing Production Readiness of Aspect Based Sentiment Analysis in the Wild

arXiv: https://arxiv.org/abs/2101.09449 · HTML: https://mlsanity.com/paper/2101.09449/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [A Baseline for Few-Shot Image Classification](https://mlsanity.com/paper/1909.02729/) (2019)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://mlsanity.com/paper/1512.01100/) (2015)
- [Aspect Level Sentiment Classification with Deep Memory Network](https://mlsanity.com/paper/1605.08900/) (2016)
- [Aspect Based Sentiment Analysis with Gated Convolutional Networks](https://mlsanity.com/paper/1805.07043/) (2018)
- [Attentional Encoder Network for Targeted Sentiment Classification](https://mlsanity.com/paper/1902.09314/) (2019)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://mlsanity.com/paper/1805.01086/) (2018)
- [Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks](https://mlsanity.com/paper/1804.06536/) (2018)
- [Model Assertions for Monitoring and Improving ML Models](https://mlsanity.com/paper/2003.01668/) (2020)
- [Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification](https://mlsanity.com/paper/1811.10999/) (2018)
- [A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction](https://mlsanity.com/paper/1912.07976/) (2019)
- [Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification](https://mlsanity.com/paper/1908.09122/) (2019)

## Built on by (most cited first, 7 of 7)

- [Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation](https://mlsanity.com/paper/2108.01682/) (2021)
- [PASTE: A Tagging-Free Decoding Framework Using Pointer Networks for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2110.04794/) (2021)
- [PyABSA: A Modularized Framework for Reproducible Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2208.01368/) (2022)
- [Aspect Based Sentiment Analysis Using Spectral Temporal Graph Neural Network](https://mlsanity.com/paper/2202.06776/) (2022)
- [ETMS@IITKGP at SemEval-2022 Task 10: Structured Sentiment Analysis Using A Generative Approach](https://mlsanity.com/paper/2205.00440/) (2022)
- [Syntactic Fusion: Enhancing Aspect-Level Sentiment Analysis Through Multi-Tree Graph Integration](https://mlsanity.com/paper/2312.03738/) (2023)
- [GHI: Graphormer over Conditioned Hypergraph Incidence for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2605.22228/) (2026)

## Abstract (arXiv)

With the exponential growth of online marketplaces and user-generated content therein, aspect-based sentiment analysis has become more important than ever. In this work, we critically review a representative sample of the models published during the past six years through the lens of a practitioner, with an eye towards deployment in production. First, our rigorous empirical evaluation reveals poor reproducibility: an average 4-5% drop in test accuracy across the sample. Second, to further bolster our confidence in empirical evaluation, we report experiments on two challenging data slices, and observe a consistent 12-55% drop in accuracy. Third, we study the possibility of transfer across domains and observe that as little as 10-25% of the domain-specific training dataset, when used in conjunction with datasets from other domains within the same locale, largely closes the gap between complete cross-domain and complete in-domain predictive performance. Lastly, we open-source two large-scale annotated review corpora from a large e-commerce portal in India in order to aid the study of replicability and transfer, with the hope that it will fuel further growth of the field.