Why deployment matters and prerequisites
So you've built a machine learning model. It can swiftly predict whatever you've trained it to predict, but that's just half of the story. In order for someone other than you to be able to use this model, you need to make it accessible to the general public or to other pieces of software that could interact with the model programmatically, sending data to that model and retrieving predictions. In other words, you want to deploy the model. Today we'll learn how to do this using Docker and FastAPI. We'll create an API for this model and test it through an interface and programmatically. First thing we'll need is a Docker engine, and it's real easy: just go to the official website and download the version for your operating system, then install it and you are ready to go. It provides a cool app that will give you an ability to control your images and containers, but we're not going to use it — we're going to do everything through the command line.
Saving the trained model artifact
Next we want to save our trained model as a binary so that we could place it inside our container which would then load it and use this model to predict on the incoming data. So here I have a really simple script for a classification model on the Iris dataset. Basically what it does: it uploads the dataset from scikit-learn, then it uses a RandomForestClassifier model to train on the data, and then we use the joblib library to dump the model or to save it as a model.job file. So if we run this script it's going to train this model and save it to our disk. Now we're not going to need this script anymore; it's not going to be used in our container, so I'm just going to close this without even saving it and I'm going to close the Python interactive window. The only thing we need here is this model.job file. With real-life data, of course, there are going to be a few more steps with your data preprocessing and saving additional artifacts for later so that in the inference pipeline your incoming data could be transformed accordingly, but for our proof of concept this is not required.
Creating a FastAPI server for inference
Now let's create our API. I'm going to create a new Python file and I'm going to call it server. We're going to need three libraries: FastAPI to create an API, joblib to load our model, and numpy. So the first thing we want to do is load our model, and for interpreting our predictions for this particular model, the Iris model, we're going to need class names. As you can see, GitHub Copilot is really smart; it's already understanding that we want to create an app using FastAPI. So I'm just going to hit tab here and create an app. The GET method decorator tells FastAPI that the function right below is going to handle all the requests coming to this path, which is going to be the root of our web app. It will be really simple: it will just return the message, the name of our API. Now let's make a function for sending data to the model so it would predict on that data. We'll create an endpoint /predict with the POST method, and the function is going to receive the data which is going to be a dictionary. It will extract features out of that dictionary, make a prediction with a model, map the prediction to the classes that we have defined here so it would return something meaningful other than 0, 1, or 2, and return this response. That's it — this is our whole API. Nothing else we need in here, other than maybe some documentation to this predict function just to give us an example what the request should look like. I'm going to create an app folder and place the Python code and the model in there because this is our server and everything required for this server is inside that folder. One thing I forgot: we're going to need the requirements as well. So here in the root, let's create the requirements.txt. Just make sure that library versions will be the same as the ones you've used when creating your model, specifically scikit-learn, because a different version of scikit-learn in the container may not load the model correctly.
Writing the Dockerfile and container basics
Now we need a Dockerfile. Our application needs some kind of environment to run in, specifically some operating system with the Python version in there and the requirements that we have just specified, because currently it runs on your local machine where you have all of this installed, but when it's going to run as a deployed container somewhere on the server that server might not have all the necessary requirements. So in the Dockerfile we're going to specify which is going to be the operating system, the Python version, the requirements, and so forth. Dockerfile without any extensions — everything we're going to write here are basically just commands that are going to be executed when the container is created. Each subsequent command will stack on the result of the previous command and they have to be sequential. The first command is always going to be the image that you are going to use to create this container. This image is going to be pulled from Docker Hub. There are a lot of different images that satisfy different criteria. In our case we're going to use Python 3.11, which is going to pull an Alpine distribution of Linux with Python 3.11 installed. The next one is just creating a code directory inside our container and assigns it as a working directory. Since our container is an isolated environment, we're going to need to copy all the necessary files that we need the container to execute. So let's just first copy the requirements and we'll place them inside the code directory that is going to be created with the previous step. Now after our requirements are copied we want to install those requirements, so we're going to run pip install those requirements inside the container, and copy the application folder inside our working directory code. Second to last is going to be EXPOSE some port because again this is an isolated environment and in order to interact with this we need to expose some endpoint to where we can send the data or we could interact with this container. The last one is going to be the command the container is going to run after it is created. Uvicorn is a server gateway interface: it's the binding element that handles the web connections from the browser or API client and then allows FastAPI to serve the actual request. It also allows you to properly spawn many workers and scale your deployment properly. The app.server:app is really simple — it will just run the server.py file from the app directory and initialize our API, which in this case is also called app. When we set the network to host, a container will share the host network stack and all the interfaces from the host will be available to the container, and the last one is going to be the port which we have exposed here in the previous step. This means that a request to a container on port 8000 will be forwarded to the application bound to port 8000 inside the container. This is literally all the code we're going to need to serve our model through an API inside a Docker container. A good idea is to always add a README file, so I'm going to just paste whatever I created before which explains everything: how to operate this API. But we're jumping ahead, so let me close this for now. I'll put a link in the description to the repo where this code is going to be located.
Building the Docker image and running the container
Now let's spin up our container and use this model through an API. We're going to need a terminal. Make sure that you CD into this root where this app folder, Dockerfile, and everything is located. Before running the container we need to build an image for this container. The command is docker build -t and any name that you want to give to this image — I'm just going to say imagename and then dot at the end. Once you run it, especially the first time around, it's going to take some time because with this first line of code here you actually download the operating system with Python version in there and it may take a few minutes, and then it will execute all these lines one after another and you can see each step is being logged into the console. Now everything is fine; we have successfully built our image. Now we want to run this image. The command is docker run --name the name of the container that you want to create. I'm just going to say containername -p which is port 8000 and the image name is the name of the image which you have created in the previous step. It looks like we have a mistake in our code on the server side, so let's go here and append a path to the model.job in the app folder because this is actually where it's stored. Now once you save it or make any changes whatsoever to your code or to the Dockerfile you have to build it again. So I'm just going to say docker build -t imagename. Okay, now we can create a container. Since we've tried creating a container before, this container name that we are trying to use once again has already been used and we cannot use the same name. We either have to go into the Docker application and delete this container or you can also use the command line for that as well. Since we have deleted this container we can run this command once again not changing the container name here, or otherwise we could have called it containername2 or any other name and a second container would have been created. It's telling us that Uvicorn is running at your localhost port 8000. Copy this URL, open a Chrome browser, and you will see the root of your application. As you can see here it tells us the message that we have included into the GET method 'Iris model API'.
Testing the API: docs, debugging, and programmatic clients
Now we have also created a /predict endpoint. If we add /predict here, we're not going to see anything because our predict function expects a dictionary and we have not sent any dictionary here. There are a few ways you can interact with this API. The first and easiest way to test this API with a web interface is going to the /docs endpoint. We have not defined this endpoint in our program, but this is the default endpoint that goes with FastAPI, so any application you have created using FastAPI is going to have this docs where you can see all your methods. This is our GET and POST method and you can test them out; you can just open it here and send your data through this request body field. Now let's go back into our code and here we have a convenient example. This is how our data is supposed to go to the model as a dictionary with a keyword features and a list of features. I'm just going to copy this and paste it in here, execute, and it seems that we have some kind of error. If you go back into your terminal where you have launched your container you can see the details and I can see the error: one of these brackets has to be right here because we cannot reshape a list; we have to reshape a NumPy array. All we need to do is replace this bracket over here and we're going to have to close this container and build it once again and then run it. We're not going to go into the Docker app to remove the container; we'll just give it a different name, so containername2. Good thing this error happened because here you can see a simple example of debugging this application. Okay, the container is running. I'm just going to refresh, go to the /predict method, try it out, paste our features in here and click execute. Very good: response code 200. This means everything happened correctly and this is our predicted class 'virginica'. The real intention usually for model deployment is to use it programmatically because say you have some data in some database that gets collected there and every once in a while you need to send some data to some model for prediction. This is where API helps. Now let's just create a simple inference script. We'll call it client.py and we'll use this to send the data from Python to an API and receive response. Let's define our data and just going to use the same sample we've plugged into the web interface. The endpoint location, convert the data into JSON format and send it to this URL as a payload using the requests library and store the reply in the response variable and I'm going to print this response. Let's check it out — 'virginica'. Of course you may have much more data; say this is an extract from your database that got collected overnight and you want to classify it, so just make a simple loop to run through these data points, create a payload out of each one of them, send it to the model and then receive your predictions and store them in the predictions list. Here are your predictions. You can also interact with the model via curl request and here I have an example in the README file. Just through terminal paste that request with your features and the URL of the model and its endpoint and receive your predictions.
Wrap-up, next steps, and further learning
We have successfully deployed a machine learning model into a container, created an inference API, and made it runnable on our local machine. This is a good start. Deployment to the cloud involves quite a few more things like setting up AWS or Kubernetes or DigitalOcean, writing scripts that interact with your repository and push your code onto the hosting environment, scalability issues, and other things. If you want to dive deeper into the subject, below I will link the best Coursera and Udemy courses. Stay tuned.
