Use Case: Creating a Resume Knowledge Base
Hello everyone, this is Professor Pattern and in this video, we're going to be looking at the knowledge base section of our Open WebUI workspace along with the document settings for our admin panel.
Now let's start with a very simple use case. Imagine that you are a recruiter for a company and you put out a job and you had a bunch of different candidates apply for it, right? You downloaded all of their resumés, some that are in PDF format, some that are in Word format. Now what you want to do is find out who is going to be the right candidate for the role. What you can do is create a knowledge base.
So the way that you would do that is from your home screen, click on workspace and then select knowledge. Here I'm going to select this plus icon and you can fill it in with 'Candidate Résumé Pool' and this is a collection of rsums that I have collected from some candidates.
Now, let's create the knowledge base and then I'm going to add this content by uploading a directory. So I'm simply going to select the directory where I've downloaded all of the resumés. Once I do that you can see all of them have successfully loaded in.
Querying the Knowledge Base with a Job Description
Now suppose that I want to find a candidate for a particular role. Let's go to our home window. Here I'm going to take a prompt, and the prompt is 'the right candidate' and basically it says based on the job description provided, take a look at the knowledge base and tell me who would be the best candidates for the job.
So the first thing I need to do is provide it the knowledge base, so 'Candidate Resume Pool'. So now it knows it needs to look at this knowledge collection. And then let's bring up a random job description. So I found one for a data scientist from OpenAI website. So I'm going to simply select all and then copy this, and then I'm going to paste it here on the Open WebUI window. Let's run this.
And we can see here that this is a data scientist role and it requires someone with 10 years of experience and some experience with Python and R, some ETL workflows, Mode, Tableau, Looker. So here we can see that based on this job description as well as requirements, it selected Alexander Monroe and Jessica Lawson as the two candidates that would be the best fit for this role. It said that both candidates have strong qualifications, but based on that, let's see if it said... okay there we go. Alexander Monroe slightly edges out as the optimal candidate.
So we can even take a look at Alexander Monroe's resumee. So this is what it looks like, and we can see that they are a data scientist with 10 years of hands-on experience so that's perfect.
Let's also try this with one more job description. So this is a job for a recruiting manager. So again, I'm going to simply copy this and then let's go back here and say 'find me the best candidates for this job'.
Okay, so here it said that based on the recruiting manager requirements, Peter Thomas does seem to have a good amount of experience. It says they have 18 years of experience in senior recruitment roles. And let's confirm that by looking at Peter's resume. So Peter Thomas, I think that's the one. And if I open this up, we can see that this is a person that has had a lot of experience with recruiting and seems like they are going to be the right candidate for the role.
How It Works: Introduction to Embedding Models
Now this works great, but how does it actually work behind the scenes? Well, to do that, let's go over to the admin panel, select settings, and then select documents.
Now the first thing that stands out is something that's known as the embedding model. Now I have another video that documents exactly how AI models actually understand human text, but if you don't want to become an expert on natural language processing then you don't need to watch that video. If you do, I've linked it in the description. All you need to know is that the goal of an embedding model is to take the text that we provide and convert it into a numerical representation. The way that embedding models are trained is by providing it a sentence. So for example, you can say something like, 'Mitochondria is the powerhouse of the cell.' Now when you provide this sentence, the goal of the embedding model is to look for each word and all the words that are similar to that word. So when it sees 'cell', it now understands that 'powerhouse' and 'mitochondria' are going to be words that are usually mentioned in this context of 'cell'.
Now just like how we provided this sentence, we would give it a huge collection of documents, maybe the entirety of Wikipedia, and it's looking at every single word and all of the context that that word is mentioned in. Now at the end, you get something like this. It looks scary at first but this is just an embedding projection. So if you click on any of these words like for example, 'systems', we can see all of the words that are similar to 'systems'. And all of these, when I say similar, what I basically mean is that they are all numerically close to each other, to the word 'systems'. If I say for example, 'Karachi', we can see that words that are similar to Karachi are 'Sudan', 'Cyprus', 'Algeria'. If you take the word 'constitution', words that are similar to it are 'parliament', 'presidency', 'territory'. So all of these words are close to this particular word. And when I say close, I just mean like in terms of the overall distance between each point and that one word. So that is the goal of the embedding model.
Finding and Changing Embedding Models
So let's go back here and we can see that the embedding model that we selected is the Sentence Transformer miniLM L6. And we can find all of these different embedding models by going over to sbert.net.
So on SBERT, we can see that there are a bunch of different pre-trained models. Now each model is trained a little bit differently, and so far we can see that the miniLM is a model that's really fast and the size of this model is going to be pretty small compared to all the remaining models. Now suppose that you want to try out a different model. Well, all you will do is just go over to that model page. So suppose that you want to try this one, all-miniLM-L12-v2. If you select this link, we can go here and then we can simply copy this. Now once we copy this, we can go back here to Open WebUI and then simply paste that. So it's the sentence-transformers/all-miniLM-L12-v2. And then once you do that, you can simply download this and then you can start using this model.
Exploring Different Model Types
At this point you're probably wondering, well what's the difference between all of these different embedding models? Well, they're all trained slightly differently, right? They have their own sort of flavors. Maybe some are going to be trained on large amounts of text. Maybe some are trained at the sentence level. Maybe some have very specific vocabulary terms based on a domain, for example, healthcare or aerodynamics or finance.
Now you can search here on sbert.net. You can see that there is the Massive Textual Embeddings Benchmark leaderboard. So this is what we call the MTEB leaderboard set, and it tracks the performance of all of these different models across a bunch of different types of tasks. Now another way in which you can also search for these models is just click here on 'Spaces' and then select 'Models'. Now here select libraries and then search for sentence-transformers. Now if we sort by maybe most downloads, we can see that sentence-transformers all-MiniLM-L6-v2, that's the one that's by default available on Open WebUI.
Now if you want to try a different model, we can see that we can maybe try the multilingual miniLM model, for example, if you have different languages that we are working with. If we have maybe medical specific vocabulary, we have this one. It's the Nomic PubMed based embedding. So PubMed is just a place where we can search for medical terminology. So we have a lot of medical or healthcare based papers. And if you scroll down we can see the overall performance of this model compared to the all-miniLM-L6. So this one does perform a little bit better for medical related tasks.
If we go back here, we can see that we also have something like Nomic Embed, and we also have Jina embeddings. Now these are going to be in terms of licensing a lot more permissible. So a lot of people like to use these models as well. Suppose that this is the model that I want to try out. I'm simply going to copy this, go back here to my admin panel and then paste it. Let's select download and it's going to say that it's going to reset the knowledge base. Based on the warning that it says if you update or change your embedding model, you'll have to re-import all of your documents. And that's fine because we are simply uploading a folder. So it said embedding model is now changed to nomic-embed-text-v1.5 and if I go back over to my workspace and knowledge, we can see here that all of these resumés still exist. So all we would need to do is just simply select 'Add content', 'Upload directory', and then select the folder and that way it will get updated.
The RAG Process: Chunking and Top-K
Now let's go back here to the admin panel, settings and documents. So what's actually happening under the hood? Well, the goal of the embedding model is to take all of our texts and convert them into a numerical format. Now when we ask a question, say something like 'find me the best candidate with SQL and Tableau experience', well first what we'll do is convert that query into an embedding representation and then what we want to do is simply find the most relevant pieces of text related to that question.
So similar to what we had here, all we will do is just find... instead of words, we're looking for sentences or chunks. So our goal is to essentially provide the large language model the query along with the five most relevant chunks. Now we can select the amount of chunks based on this top K parameter. So if you wanted to say like, okay, give me the five closest sentences or the 10 closest chunks of text based on this question, then that's what the top K parameter is for.
Now, if you scroll down, you can see the size of each chunk. So it says that the size is a thousand. Now this again depends on whatever embedding model you use, whether it's looking at a thousand characters, words or sentences. And then the chunk overlap is how much overlap there is between each chunk such that it retains some of the context.
Advanced RAG: Prompt Templates and Hybrid Search
Now, if you go here, the magic actually happens in the RAG template, right? We specify that the task is it needs to respond to the query. We gave it some guidelines along with some examples of what a citation looks like. Now the output is provide a clear and direct response and use this based on the context. Now the context that we provide to it are going to be the three most relevant chunks based on the text. And the query is whatever the user query was.
Now, suppose that you wanted to change some of these parameters. Suppose that you wanted to have more context, right? You want to also understand it from the LLM perspective. Maybe some models are going to have a huge context window, something like Claude 3.5 Sonnet that has a context window of 200,000 words or tokens. So you want to be careful if you select a larger context size, then maybe you might end up filling up all of that memory.
Now here if you scroll up, we can also see that there's something else that's known as hybrid search. So what happens if we enable this?
Now if we previously were looking at just the embedding model, what we were doing was simply looking at the closest pieces of sentences or chunks related to the query based on where they lie in this embedding representation. However with hybrid search, we also have two different parts. We have the embedding model, but we also have something that's known as the reranking model. Now all this is, is basically another way in which we can find the most relevant chunk based on our query. So here for example, we can set for a reranking model as a default one, or if you wanted to we can go back here to the MTEB leaderboard set and then search for specifically reranking. So it would be right here, reranking, and then we can search for one, maybe like that's the top of the leaderboard. So let's go back here. When we enable hybrid search what it's going to do is that it's going to find the top three most relevant chunks based on the embedding model, and then the top three most relevant chunks based on the reranking model. And then based on these two searches is going to give us the three most relevant pieces of text. So it's just another way in which we can really improve our process of getting the most relevant pieces of text that our model can compare with.
Conclusion
So that is going to be the overall idea behind the document section of the Open WebUI interface. Hopefully it gave you a little bit of an understanding about how all these components are kind of related to each other. If you wanted me to do a deep dive into any of these components or wanted to understand some things in more detail, let me know in the comments and I'll definitely make another video on that. Thank you all for tuning in. I'll see you in the next one. Have a good one.
