Introduction & Final App Demo
Hi everyone, I'm Patrick, and in this tutorial, we learn how to deploy machine learning models with FastAPI and Docker and then have a production-ready app. So you can use this template to deploy the container everywhere you want. In this video, we go ahead and deploy to Heroku because there's a free tier and you can follow along. Also, this approach should work in the same way for any machine learning or deep learning framework you want to use. So if it's scikit-learn or TensorFlow or PyTorch, the approach is the same, and it's also not too difficult. So let's get started.
Let's quickly test the final app. In this video, we built a language detection model and as you can see here, we have the live URL at heroku-app.com/predict, and we send a POST request with this text. If we click on Send, we get this response with languages: English. If I, for example, switch this with the German version and click on Send, then the language should be German. So this is correct.
Fun fact on the side, at AssemblyAI, we also have an automatic language detection feature in our API. So this is actually a real-world project. So let's see if we also can develop an accurate model and deploy this.
Training the ML Model in a Notebook
First, let's build and train the model in a notebook. So here I'm in a Google Colab, and the focus of the video is not how we develop this specific model, but rather how we go from notebook to production app in a moment. So let's go over this very quickly. The data set is available on Kaggle. Also this notebook and all the rest of the code is available on GitHub. I put the link in the description. So here we have our imports. Then we load the dataset and analyze this. So we have lots of different texts and the corresponding language. This is our X and Y.
Then the first step we apply is a Label Encoder. So this transforms the labels 'y.' Basically what it does is that for each of these texts, these classes, it assigns a number starting from zero. And whenever we have a preprocessing step that transforms our data, we have to remember this and later also use this in our code.
So for example, this is another pre-processing step. Here we apply regular expressions to remove these special characters. So we also have to use this in the code later. Then we have the typical train-test split, and now we build our model. It consists of two parts in this case. The first one is a Count Vectorizer. Here we fit this and then transform this. And then we have the second step, which is a Naive Bayes model. And then we can use model.predict with the test data. And then here we have some metrics and for example, print the accuracy. So we can see this is 98%, so pretty good.
Now, one best practice we can apply here, if it's possible, is to combine all steps into only one step. This is much less error-prone, and then we only have to save one model instead of two here. So with scikit-learn, we can do this with the pipeline. With TensorFlow, for example, we often have a sequential model where we can put in all the layers in this sequential model. So with the pipeline, we can now combine the vectorizer and the Naive Bayes. And then we can fit the pipeline, and then we can use this pipeline to predict the test set. So now we only have to apply this one step. And if we compare the accuracy, then we see this is the exact same result.
Saving and Versioning the Trained Model
So now when we are done with training the model, we have to save and download this. And with scikit-learn we can do this with pickle.dump. With TensorFlow and PyTorch, there's also a very simple API to save your model. One thing I recommend to do here is to also save a model version so you can keep track of the current version you have. So we use this syntax: major, minor, and patch version. So this is our first minor model version. And one other thing I want to mention here is that now in this case, this will be one Pickle file. So we can actually go ahead now and click on download here. But if you use TensorFlow or PyTorch, then often it will save this in one whole directory and we cannot download a whole directory. So there's no download button. So one trick you can apply is to zip a folder with this command, so: !zip and then you give it the name of the zip file and the name of the folder, and then you can download the zip file.
So now we have this and finally, let's test this one more time. So here we have the text, and if we run this, then we see this is Italian. And we can also see that Y is only this number eight here. So here we actually apply the label encoder classes to get the actual language: Italian. So I printed this in the top. Here are the label encoder classes. So we also have to get this for our code. And yeah, now we're done. So now we can use this model and build our app.
Creating the FastAPI Application
Now let's create the FastAPI app. For this, in the root directory, let's create an inner directory app, and in here we will put all the code. So we have one main.py file, this is where we will put the FastAPI endpoints, and I go over this in a moment. And then we have an inner model directory, and here I store the downloaded pickle file, the trained model, and then another file that I called model.py. Basically, this is a helper file that does the model prediction.
So here I hardcoded the model version. Then I also use path and the path of the current file to get the base directory. And this is because the folder structure inside the Docker container can be a little bit different, and I want to make sure that we can find this pickle file here. So then we can open this and make sure that this version corresponds to the file name, and then we can say pickle.load and load our model. Then here we have the classes. So this is from the label encoder, the classes in the same order. And then we only need one helper function in this case, predict_pipeline, that gets the text. So this is a string, and then here we do all the same pre-processing steps that we did in the notebook. And then we can say model.predict because we now have only one step with the pipeline object. So then we get the prediction and this is a number. So then we have to access the index of the classes, and then we can return the language. So this is the model.py.
And now let's go to the main.py. So here we import FastAPI and BaseModel from Pydantic. I'll show you what this is doing in a moment. Then we also import predict_pipeline and the version as model_version. And be careful to start your path with app, so app.model.model for this file and then it also finds it in the Docker container.
Then we create the app, and in FastAPI, it's super simple to create your endpoint. So it's very similar to Flask. We define a function and then decorate this with @app.get or @app.post. So I often like to have an endpoint for a health check. So here we simply return health_check: "OK". And here also the model version for additional information. For example, you can also return the API version here.
And then we have one predict endpoint. This is a post endpoint, and in here we simply call predict_pipeline and put in the text and then we return the language. And this is a dictionary. And now to make sure that we pass the correct data types to this API when we send the data, the input we want to have should be a string. And FastAPI works with type hints. So for this, we can define a class TextIn that inherits from BaseModel, and this should only have one field. This is a text that should be a string. And now when we send the data and this is not a string, then FastAPI can detect this for us automatically and then raise an error or show an error in the API. And this is a super cool feature of FastAPI, so this is why we use this BaseModel.
And in a similar way, we do this for the output. So for the response, we say PredictionOut which inherits from BaseModel, and here this should be one field, language, and this should also be a string. And then in the code here, we can access payload.text, and we have to make sure that we have one field language. And this is basically all we need for this simple API. And now we can Dockerize it.
Dockerizing the Application with a Dockerfile
Now to Dockerize this, of course, you have to have Docker installed on your machine. And then in the root directory, we need to create a file that is called Dockerfile. And then it's also a good practice to have a .dockerignore. This is similar to a .gitignore. Here I simply copy-paste this from GitHub, you find this in the description, and this ignores certain files inside the container.
And now for the Dockerfile, we can go to the official FastAPI docs. There's one section, FastAPI in containers. So I recommend reading through this because there are different ways of doing it. One way of doing it is to use the official Docker image with Gunicorn-Uvicorn. You can also find this here on GitHub. So it says Docker image with Uvicorn managed by Gunicorn for high-performance FastAPI web apps. And in order to use this, we can copy this code. So this goes in the Dockerfile, and this is basically all we need to do. And this uses this base image. Then it copies the requirements.txt inside the container. Then it runs pip install requirements, and then it copies the app directory also in the container. And that's why also we have to have this folder named app.
And then we need one file that is called requirements.txt. And here we put in all the dependencies. So in our case, since we already used the base image, it already includes FastAPI, Uvicorn, and Gunicorn. So the only missing dependency in our project is scikit-learn. For example, if you use TensorFlow, then you can put in TensorFlow here, or PyTorch.
And for example, it's also important to mention or worth mentioning that you can use tensorflow-cpu oftentimes because you don't need the full version, and then your container will be much smaller. So then it's also good practice to pin the version. So for this, we can go back to the notebook and then in here we can for example say import sklearn and then sklearn.__version__. This should give you the current version. And then we can copy this. So it's 1.0.2. So it follows the same pattern: major, minor, patch. So let's go back and say == and then this. And now this is all that we need. So now we can build the image.
Building and Testing the Local Docker Container
Now to build the Docker image, we can open a terminal. And then I actually noticed one small change that we have to do. So as you can see in the Dockerfile we have the app/requirements.txt. So basically an app directory is the new root directory and we have an inner app directory. So we have to say COPY from app to app/app. And now save this.
And now we can say docker build -t and give it a name. Here I call this language-detection-app, and then a dot for the current folder, and then hit enter. And now this will build the Docker image. And now this was already done. So if you do this for the first time, then this might take a few seconds or minutes. So now we have this and now we can run the container by saying docker run and then we map the port from the Docker container to our machine by using 80:80, so this is port 80, and then the name language-detection-app. So now this is running and starting and now you should see that the Uvicorn workers are starting and it's listening to 0.0.0.0 port 80. So basically this is our localhost port 80. So now we can find this and then go to this route. And for the base route, this is a GET request. So this is working: health check okay and model version is 0.1.0.
So now we can for example use Postman to send a POST request, or what's really cool with FastAPI is that we get automatic documentation by using /docs. And here we can see all the endpoints. So we have the home and we have /predict. And then we also can try this out. And as you can see, it knows that we need this schema with a text field and then the string. And this is because if we go back, here we defined this as a BaseModel with text, so the text and it needs to be a string. So that's why it knows that we need these fields. And then we can click on 'Try it out' and let's say 'hello, how are you?' and then execute. And then we get the response directly here and can see language: English. And you also get the curl command if you want to try it from curl. So let's try it with a German sentence. So let's say, 'Hallo, wie geht es dir?' and then execute this. And then language is German. So this is working. So now as a last step, let's deploy this to Heroku.
Deploying the Container to Heroku
And now we can deploy the container everywhere we want. In our case, we do it on Heroku. So let's actually stop our local container and clear this. And now the first thing we want is a Git repository. So we can say git init. And of course, you have to have Git installed on your machine. This will initialize an empty Git repository. Then I also want to add a .gitignore. So here we also ignore some files and folders we don't need, so I only ignore the virtual environment and this file. And now you could continue in this terminal. I actually want to switch to my normal one. So here we now say git add ., so everything. So we can check git status that all these files have been added. And now we can say git commit and give it a message. So let's say 'initial commit'.
And now we can start creating our Heroku project. So for this, of course, we need to have a Heroku account and the Heroku command line interface installed. And now we can do everything from the command line. So first we say heroku login, and now this should open the browser. And then here you should be able to put in your credentials. So I think I already did this and this should be stored. So yeah, so now we can go back and see we are now logged in.
And now before we can continue, we actually need one more file, also in the root directory, and this is called heroku.yaml. And for this, I can recommend a documentation site on the official Heroku Dev Center, 'Building Docker Images with heroku.yml'. So basically, we only need this part. So let's copy and paste this in here. Then of course, we have to add this to Git again. So we say git add heroku.yml, and then we say git commit and 'add heroku.yml' as message.
And now we can create a new Heroku project. So we say heroku create and then we have to give it a name. So let's use language-detection-app and then it needs to have a unique name that is available. So let's try 12 and see if this works. All right, so this worked. So now this will be the URL that we can use to access our API.
So now we can say heroku git:remote and then we set the remote for this project, so this was called language-detection-app12. And now this will create a remote. And now we can say heroku stack:set container, because this is using a Docker container. And now we only have to push this with Git.
So now we can say git push heroku main, but as we can notice, we are still on the master branch. So we have to say or have to change the name of the branch to main by saying git branch -m main. Now we are on the main branch, and now we can say git push heroku main. And now it will push everything to Heroku and start the app. So this might take a while. All right, and this worked. So deployment is done.
So now we can grab the URL we've seen above. So this one is where our API is now live. So this time I want to test this with Postman. So we can check the home URL with a GET request and send this. And then we see 'health_check: OK' and model version. And now we send a POST request with this data to /predict. And now let's see. Language is German. So now let's check a different text. So now let's say 'ciao'. And now let's send this, and we get Italian. So this works.
So this is our app deployed at this endpoint. And this is all that I wanted to show you. I hope you really enjoyed this tutorial. If you have any questions, let me know in the comments below. And then I hope to see you in the next video. Bye.
