# Hierarchical Classification of Financial Transactions Through Context-Fusion of Transformer-based Embeddings and Taxonomy-aware Attention Layer

arXiv: https://arxiv.org/abs/2312.07730 · HTML: https://mlsanity.com/paper/2312.07730/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 12 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 3 of 3)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Tabular Transformers for Modeling Multivariate Time Series](https://mlsanity.com/paper/2011.01843/) (2020)
- [Merchant Category Identification Using Credit Card Transactions](https://mlsanity.com/paper/2011.02602/) (2020)

## Built on by (most cited first, 1 of 1)

- [Using Zero-shot Prompting in the Automatic Creation and Expansion of Topic Taxonomies for Tagging Retail Banking Transactions](https://mlsanity.com/paper/2401.06790/) (2024)

## Abstract (arXiv)

This work proposes the Two-headed DragoNet, a Transformer-based model for hierarchical multi-label classification of financial transactions. Our model is based on a stack of Transformers encoder layers that generate contextual embeddings from two short textual descriptors (merchant name and business activity), followed by a Context Fusion layer and two output heads that classify transactions according to a hierarchical two-level taxonomy (macro and micro categories). Finally, our proposed Taxonomy-aware Attention Layer corrects predictions that break categorical hierarchy rules defined in the given taxonomy. Our proposal outperforms classical machine learning methods in experiments of macro-category classification by achieving an F1-score of 93% on a card dataset and 95% on a current account dataset.