# A Financial Service Chatbot based on Deep Bidirectional Transformers

arXiv: https://arxiv.org/abs/2003.04987 · HTML: https://mlsanity.com/paper/2003.04987/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Feb 2020.

## Built on by (most cited first, 10 of 10)

- [Fingerprinting Fine-tuned Language Models in the Wild](https://mlsanity.com/paper/2106.01703/) (2021)
- [TaDaa: real time Ticket Assignment Deep learning Auto Advisor for customer support, help desk, and issue ticketing systems](https://mlsanity.com/paper/2207.11187/) (2022)
- [Machine Learning in Finance-Emerging Trends and Challenges](https://mlsanity.com/paper/2110.11999/) (2021)
- [Exploring Robustness of Prefix Tuning in Noisy Data: A Case Study in Financial Sentiment Analysis](https://mlsanity.com/paper/2211.05584/) (2022)
- [DataChat: Prototyping a Conversational Agent for Dataset Search and Visualization](https://mlsanity.com/paper/2305.18358/) (2023)
- [Optimal word order for non-causal text generation with Large Language Models: the Spanish case](https://mlsanity.com/paper/2502.14451/) (2025)
- [Scaling Laws for Discriminative Classification in Large Language Models](https://mlsanity.com/paper/2405.15765/) (2024)
- [A Conditional Generative Chatbot using Transformer Model](https://mlsanity.com/paper/2306.02074/) (2023)
- [Predictability and Causality in Spanish and English Natural Language Generation](https://mlsanity.com/paper/2408.14283/) (2024)
- [Staged Linguistic Seeding: Grounded Query Expansion for Verified-Unit QA in AI Contact Centers](https://mlsanity.com/paper/2609.00844/) (2026)

## Abstract (arXiv)

We develop a chatbot using Deep Bidirectional Transformer models (BERT) to handle client questions in financial investment customer service. The bot can recognize 381 intents, and decides when to say "I don't know" and escalates irrelevant/uncertain questions to human operators. Our main novel contribution is the discussion about uncertainty measure for BERT, where three different approaches are systematically compared on real problems. We investigated two uncertainty metrics, information entropy and variance of dropout sampling in BERT, followed by mixed-integer programming to optimize decision thresholds. Another novel contribution is the usage of BERT as a language model in automatic spelling correction. Inputs with accidental spelling errors can significantly decrease intent classification performance. The proposed approach combines probabilities from masked language model and word edit distances to find the best corrections for misspelled words. The chatbot and the entire conversational AI system are developed using open-source tools, and deployed within our company's intranet. The proposed approach can be useful for industries seeking similar in-house solutions in their specific business domains. We share all our code and a sample chatbot built on a public dataset on Github.