Tools to Put Deep Learning Models in Production

While there are a lot of machine learning frameworks and libraries available, putting the models in production at large scale is still a challenge. I’d like to talk about how we took on the challenge of supporting the data scientists with their efforts by making it easy to put their models in production. I’ll be covering how we:

• chose our tools and developed the internal deep learning infrastructure
• train our models in docker containers
• automate the re-training of models
• deploy models using Kubernetes

I’ll also talk about how we optimize our model prediction infrastructure for latency or throughput depending on the use case.