11-09, 11:00–11:45 (America/New_York), Winter Garden (5th floor)
This talk is a gentle guide for MLOps engineers or data scientists who have basic knowledge of Docker to build a scalable machine learning service on Kubernetes and Knative (serverless technology). It also covers the challenge of setting up a model storage for Kubernetes and how Mlflow can be used to solve the problem.
When I was assigned to deploy my machine learning (ML) model as a service, I learned Flask and Docker for the first time. After I completed my task, I was curious about how to scale up my application. In this 30-minute talk, I will share the lessons I learned by deploying my ML service on Kubernetes. You will learn about the problems of deploying multiple ML services on Docker in production, how Kubernetes solves those problems, how to create a serverless service on Kubernetes using Knative, and how to set up a model storage for Kubernetes using Mlflow model registry.
This talk consists of (i) the challenges of building a highly available and scalable system (3 minutes), (ii) a gentle introduction to deploying ML service on Kubernetes (10 min), (iii) the benefits of serverless technology and how to build one with Knative (7 min), (iv) a challenge of building a model storage for Kubernetes and how I used Mlflow model registry to solve it (7 min), (v) how I set up my 3-node Kubernetes cluster and Mlflow model registry for $30/month on Digital Ocean (3 min).
This talk will help MLOps engineers or data scientists who have basic knowledge of Docker to build a reliable and scalable ML service with open source software in production.
Previous knowledge expected
Kei currently works as a data scientist in the healthcare field. He uses his expertise in data science/software engineering to automate machine learning workflows at scale. He has a Master of Science in Data Science degree from the Graduate Center, City University of New York, where he extensively focused on deep learning for information retrieval. He is passionate about learning new technologies to achieve what was impossible yesterday.