Introduction to Hugging Face
If you are building AI apps, you have to learn how to use Hugging Face. It is one of the top AI companies, valued more than 2 billion dollars. It has more than 16,000 followers on GitHub. Its product is used by Google, Amazon, Microsoft, and Meta, with more than 200,000 different types of AI models, including image-to-text, text-to-speech, text-to-image and many more. That's why if you are building AI apps, you absolutely need to learn how to use it. And I'm going to show you how can you use Hugging Face platform and build it with other public library like LangChain. Let's get to it in the show.
Hugging Face is a place for you to discover and share AI models. So there are three parts of the Hugging Face platform: Models, Datasets, and Space.
Firstly is Models. This place where you can find all different sorts of models to use. For example, if we are interested in using image-to-text, I can select the category on the left, and then on the right side choose any of the popular image-to-text models. And once I get into this page, on the left side, they will have some description about model and on the right side it allows you to preview and test the AI model directly on their hosted version. And this is why Hugging Face is so useful. So without it, you will need to find the models, download it to your local machine or host somewhere, and then try to run it to know if it is the right model for you. But with Hugging Face, they are hosting on their own machine and you can test it immediately. But for this image-to-text model, I can drag and drop an image directly and see what kind of results it will get. If you want to use it, it allows you to easily deploy this model on different servers. You can also use hosted API on Hugging Face hub for free. It is a bit slow and they have rate limits but it's definitely enough for you to run some tests. But on the other hand, if you prefer to run the models locally on your own machine, you can also use their Transformers library. And I will talk about how do we do that very soon.
Besides models, on the outside they also have Datasets. And this is where you can find a lot of data sets that you can use to train your own model. For example, if I want to build my own voice model, I can filter down to text-to-speech and find a specific language that I want to use. Then you can click on any of them and preview what are the data sets they have. Unless you are training your own model, you probably won't use this data sets too much.
And the last part is Space. Space is initially designed for people to showcase and share the AI apps that they build. So they allow you to deploy the apps that you have been building very easily on their own machine, and they provide a free version too. But on the other side, you can explore what other AI apps that people are building. And there are a lot of very cool stuff. You can just click on them and start playing with those apps. And you can also learn how did they build those things. Clicking on this button it will show you all the models used to build these apps, and you can click on the files to see the source code.
Project Intro: Image to Audio Story
How are we going to use those models on Hugging Face while we are implementing in LangChain? I'll take you through a step-by-step example of implementing such an AI app where I can upload the image and then it can automatically turn it into an audio story.
The man and woman sat on the couch, lost in silence. He broke it, "I love you." She smiled and said, "I know."
As through this example, you will learn how to use a few different Hugging Face AI models. Let's get to it.
Step 1: Setting Up the Image-to-Text Model
Firstly, let's think step-by-step how we're going to implement this. This app will have three components where first they need an image-to-text model to let the machine understand what is the scenario based on the photo. And then we will use a large language model to generate a short story. And in the end, we'll use a text-to-speech model to generate the audio story.
And to find the right image-to-text model, we can go to Hugging Face and filter down the image-text models. The one I will be using is the one called BLIP. You will need to create a Hugging Face account and then go to settings, access tokens, and create an access token for LangChain.
Let's go back to the Visual Studio and create a .env file where we store all the credentials, and I'll put Hugging Face Hub API token. Once I save that, let's import a few libraries. We'll first import dotenv and run this to be able to access the Hugging Face API token that we store in the EMV file. And then we will import pipelines from Transformers. So pipeline will allow us to download the Hugging Face model into our local machine. And now we're ready to implement the first one: image into text model. We've created pipeline to load the AI model. Firstly, we will put in the task here, which is 'image-to-text'.
Running the Model and Understanding Tasks
Some of you might be curious where do we get this task name 'image-to-text' from. So Hugging Face Transformers library actually has a predefined list of tasks and you can go to this URL, huggingface.co/tasks, to understand what are the tasks that it supports. And you can click on any of them to get a more detailed tutorial about how to use that specific task.
And then we need to put the model name. You can get the model name by clicking on this 'Use in Transformers' and just copy paste this one. And we will run image_to_text passing the URL of the image file. Then we're going to print it. I'll copy paste a photo in the root folder for the testing purpose. Now let's see what results we got. We run python app.py. Now I got this result: 'a group of people standing on a boat,' which is a very accurate description. And I only want to return the actual text here, so I will choose the library of the first item and 'generated_text'.
Step 2: Generating a Story with LangChain
And next is that we want to use a large language model to generate a short story based on this scenario that we got from the image. You can use some open source model from Hugging Face as well, but for me, I do prefer to use GPT, so I'll use LangChain here. So let's add in the OpenAI API key here as well. And then let's import a few libraries from LangChain. And this is a function we're going to run. We will firstly create a prompt template that lets GPT to generate a story. And then we'll create this LLM chain with GPT 3.5 turbo. Let's try. And now all we need to do is just turn those tags into a speech using a text-to-speech model.
Step 3: Text-to-Speech via the Inference API
And again, we will do the same thing, go to the models page, find text-to-speech models and find the most popular one. But this time, I want to share another way you can use Hugging Face models. So you can click on this deploy button and there should be an option called Inference API. And this is a super easy and fast way for you to test out the Hugging Face API for free. So this is what we're going to do. We're going to use this requests library. Going back here, import requests and then create a function text_to_speech. And at the top, I'm going to load the Hugging Face API token so that I can pass it on to the API request. And I will add in the API URL here, putting the header that passed on my Hugging Face API token. And then create a message of inputs. Now just call this API request. And for the model I'm using, the result it returns is a FLAC file, which is one type of audio file, so I'm going to store it locally. Let's try it. Oh, sorry, I forgot to add the OS library which will allow us to get the API token. And let me try again. Okay, here we go. We got this audio file generated.
Building the UI and Final Demo
The group of people were standing on the boat, their eyes fixed on the horizon. The sun was setting, painting the sky with brilliant shades of pink and orange.
Okay, now you can see the whole thing has been working. All we need to do now is just connecting everything together and give the UI a layer with Streamlit. So I'll import streamlit as st, which is the library that allows us to create a user interface for python code. At the bottom I will add this and also create a main function which will be called when the app is loaded. So first I will set the page title and then it will give a header, 'Turn image into audio story'. I will put an uploaded file equal to st.file_uploader. This will allow people to upload an image file. Then if the file is uploaded, I will firstly try to save this image and display the image by st.image. Then I will call in the function that we created, get the model to generate text from the image we uploaded and then let GPT generate a story based on the scenario. In the end, our generate audio file from the stories and we're going to display the scenario and stories in here. And in the end, we're going to display the audio file that we got. And that's pretty much it. Let's try to run this app by doing streamlit run app.py. Okay, so here we can upload the image and we can see it is running here. If we open the terminal, we should be able to see what it's doing now. Okay, here you go. You can see we already generated the scenario and little story here. If you click this play button, it should play the audio file.
As they sat together on the couch, the man stared intently at the woman he had known for 40 years, and every time he looked at her, he felt like he was seeing her for the first time. Suddenly, he blurted, "I think I love you." The woman turned to him, her eyes wide with surprise, and then with a smile that lit up the room, she said, "Finally." From that moment on, they knew they were meant to be together.
Recap: Two Ways to Use Hugging Face Models
Okay, I think this is a pretty dope use case. So this is how you can use Hugging Face models. To quickly recap, the easiest way for you to use that would be you can use the Inference API to use their hosted version directly. On the other side, you can also use pipeline to download all those models on your local machine. If you want to learn more, I highly recommend you go to the hf.co/tasks to learn all the different type of tasks it supports as well as different type of models it has.
Low-Code Alternatives & Conclusion
One last thing I want to touch base on is I realized one of the low code AI chain builder platform called Relevance AI, they actually provide image-to-text model out of box, and I can just create this image-to-speech app super quickly with their local UI and get a deployed app out of box in just like five minutes. I do hope they can build a deep integration with Hugging Face where I can just grab different type of AI models directly, but it's already a pretty good start, so highly recommend.
Alright, hopefully you know how to use Hugging Face now and start building some super interesting AI apps. If you found this content useful, please subscribe. I'll continue sharing all those AI experiments I'm doing. Thank you and see you next time.
