# Responsive parallelized architecture for deploying deep learning models in production environments

arXiv: https://arxiv.org/abs/2112.08933 · HTML: https://mlsanity.com/paper/2112.08933/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools](https://mlsanity.com/paper/1903.11314/) (2019)
- [Hierarchically-Refined Label Attention Network for Sequence Labeling](https://mlsanity.com/paper/1908.08676/) (2019)
- [Serving deep learning models in a serverless platform](https://mlsanity.com/paper/1710.08460/) (2017)
- [End-to-End Resume Parsing and Finding Candidates for a Job Description using BERT](https://mlsanity.com/paper/1910.03089/) (2019)
- [Parallelizing Machine Learning as a Service for the End-User](https://mlsanity.com/paper/2005.14080/) (2020)
- [Microservices in Practice: A Survey Study](https://mlsanity.com/paper/1808.04836/) (2018)
- [Component-aware Orchestration of Cloud-based Enterprise Applications, from TOSCA to Docker and Kubernetes](https://mlsanity.com/paper/2002.01699/) (2020)

## Abstract (arXiv)

Recruiters can easily shortlist candidates for jobs via viewing their curriculum vitae (CV) document. Unstructured document CV beholds candidate's portfolio and named entities listing details. The main aim of this study is to design and propose a web oriented, highly responsive, computational pipeline that systematically predicts CV entities using hierarchically-refined label attention networks. Deep learning models specialized for named entity recognition were trained on large dataset to predict relevant fields. The article suggests an optimal strategy to use a number of deep learning models in parallel and predict in real time. We demonstrate selection of light weight micro web framework using Analytical Hierarchy Processing algorithm and focus on an approach useful to deploy large deep learning model-based pipelines in production ready environments using microservices. Deployed models and architecture proposed helped in parsing normal CV in less than 700 milliseconds for sequential flow of requests.