Introduction: The Challenge with Custom Knowledge Bases
Flowise makes it easy to create AI applications using a clean and intuitive user interface. The true power of Flowise is the ability to create AI apps that make use of a custom knowledge base. Using our own data, we can add the ability to upload files to our application, like PDF documents and text files, and we can then chat to our documentation using Flowise.
Before we build this document chatbot, though, there are a few concepts that we first need to understand. This is a topic that I could probably create an entire video on, but let's have a look at the fundamentals for this video. In this example, we have a file containing the nursery rhyme 'Mary had a little lamb', and on the right-hand side, we have a chat app where we can also model questions. As an example, we might want to ask the assistant questions about the contents in this file. But in order for the model to have a view of the contents of this file, we first have to provide the contents of this file as context into our application, which could look like this. And that would be the same as copying the text from the file, pasting it into ChatGPT, and then asking ChatGPT questions about the file. That could work, but there is an issue with that. And the issue we have is with the token limit.
A token represents a word or a part of a word, and if the contents of a file were quite large, we will exceed the token limit quite easily. So simply copying the file into the chat and then asking questions will become an issue.
Text Splitters and LangChain Documents
So therefore, ideally, we only want to grab the sections that are relevant to our question and only feed that into the chat as context. And thankfully, Flowise, or LangChain, which Flowise is based on, offers a neat and elegant solution to work around this limitation.
So let's talk about Text Splitters. Text Splitters allow us to take the content of the file and then break the content up into chunks. Let's also talk about documents. And documents should not be confused with files, but instead, a document is a LangChain definition of a chunk of text which we got from the text splitter. But the document also includes metadata like the file name and any other information that we want to store about this piece of text. As an example, let's imagine a scenario where we want to upload a folder full of files to our application. The document loader will loop through each of those files, and then it will use the text splitter to break the file up into chunks, and it will use the metadata to store the file name that contained that piece of text.
Embeddings and Vector Stores Explained
Now that we've broken up our file content into logical pieces of text called chunks and then converted those chunks into LangChain documents, which also contain metadata, we now need to store that information somewhere. And how this works is we'll actually store that information in a database called a vector database. Vector databases are a fascinating topic, but it will take way too long to explain how they work in this video. But Vector databases basically store the data as vector arrays. A vector array is something that the AI will understand and it will assist the AI with finding similar documents when we chat to it.
But in order for us to convert our text into a vector array, we need to run a function called embeddings. Embeddings is a unique algorithm that is unique per language model and this will convert our text into something that our model will understand. Because we are using OpenAI in our example, we will use the OpenAI embeddings function to convert our text to this array. Our vector array, along with the text and metadata, can now be stored as records in our vector database. And furthermore, vector stores also group similar pieces of text or similar embeddings in clusters.
This means that in our application, if we now had to ask our application, 'Who is Mary?', our chat app will first go to the vector store to perform a similarity search. So it will go and extract all documents that are related or similar to Mary. It will then return a list of similar documents back to our application. By default, I think Flowise returns the top four results back to the application. It is then these results that get included in the conversation as context, greatly reducing the amount of tokens needed for our conversation.
Setting Up the Flowise Canvas
Enough theory, let's now go and build this document chat bot in Flowise. Go back to the dashboard and let's create a new new chat flow. Let's save this and call it 'Document Chatbot'. What we also need is a file to upload to our application. Ideally, you want to create a file with information that GPT wasn't trained on. So I used ChatGPT to generate a unique story and I added the contents to a file. Some details of the story is that this is some sort of love story about a young woman named Emily and she's an architect who moved to a small town where she met a man called Lucas. So there are some details in the story that GPT usually won't be aware of. So go ahead, create a file and save it on your machine.
So back in our project, let's have a look at what we need. As I mentioned in the previous videos, our chat flows always require either a chain or an agent to generate output. In this example, we don't have to use any external tool, so we do not need to use agents, and we will be using chains. If we go to our nodes and open up chains, we can see this chain called Conversational Retrieval QA Chain, and this is a document QA chain, and this is perfect for what we need. Let's drop this chain on the canvas and let's configure it.
Configuring the LLM and Vector Store
If we look at the inputs, this chain requires an LLM as well as a vector store. Remember in our theory, we showed that our data will be retrieved from a vector store. Let's set up our language model. Under nodes, let's go to chat models and let's drag and drop the Chat OpenAI model onto the canvas. And we can immediately connect this LLM to the chain. We also need to provide our OpenAI API key, like so. We can leave the model on GPT-3.5 Turbo.
Let's now also set up the vector store. To do this, let's click on nodes. Under nodes, we can open up vector stores, and within vector stores, we've got quite a few options. For this demo, we'll simply use the in-memory vector store. But in a production environment, you might want to consider one of these other options like Pinecone or Supabase. Let's add our in-memory vector store to the canvas and let's hook it up to our chain. Our chain now has access to an LLM as well as a vector store. So let's go ahead and load our data into the vector store.
Implementing the Document Loader and Text Splitter
First off, let's load documents into the vector store. And you might recall from the theory that this document is not referring to a text file, but instead it's referring to a LangChain document. This is basically a chunk of text with metadata. So how do we create documents? This is actually quite easy. All we need to do is add a document loader to our project. So under document loaders, we have quite a few options. We have the Cheerio web scraper, we can upload CSV files, docx files, we can even load a folder with multiple different files within it. Let's keep it simple and add the text file document loader to our project. This text file node will allow us to upload files from our machine. It will go and create documents from the content of that file. So let's hook this node up to our vector store.
But what this is going to do is it's going to upload our text file in its entirety and create one single document with metadata from that file, which is not what we want. But what we want to do instead is upload our file and then split our file contents up into chunks. And then from those chunks, we want to create documents. So optionally, we can attach a text splitter to this node. In nodes, go to text splitters. Within text splitters, we'll select the recursive character text splitter and then add that to our canvas. We can then connect this text splitter to the text splitter parameter on our text file node. We can now tell the text splitter how big these chunks need to be, and the default is 1,000 characters. Let's make that smaller by changing it to 200 characters. The size of the chunks is really up to you, but just keep in mind the intention is to grab these chunks and then add that to the context of our conversation. And the smaller the chunks, the better, because the smaller the context, the less tokens we use, which drives down costs. We can also specify a chunk overlap, which will change to something like 20 characters. This means that each chunk might have a section of the chunk before and after it available in its contents.
So now we are able to upload files by selecting the file from our machine and this will now take our file, chunk it based on these parameters, and then convert each of the chunks into a LangChain document, which is then stored in our vector database.
Adding Embeddings and Testing the Chatbot
But this brings us to the final component of our chain, and that is embeddings. In order for the AI to make sense of the content that we're storing in the database, it needs to convert the text into vector arrays. And in order to convert the text into a vector array, we need to call the embeddings function. This is quite easy to set up as well. In the nodes, we can go to embeddings and under embeddings, we can select the embedding function related to our LLM. Because we are using OpenAI as the LLM up here, we will simply select the OpenAI embeddings function and drop that onto the canvas. And we can connect that to our vector store. And in order to call this OpenAI embeddings API, we need to provide the OpenAI API key as well.
We can now go ahead and save this chat flow and we should be able to test this out. Let's open up the chat and let's ask a question specific to our file. So let's actually pull up this file to the side over here so we can test this out. We know that the main character is called Emily, so let's ask it a question, 'Who is Emily?' and let's send this. And that is perfect. Emily is indeed an architect and she is from Everdale. Let's ask it, 'Who is Lucas?' And it seems that our story does not provide enough information about Lucas. So let's rephrase the question a bit, 'Are Emily and Lucas friends?' And indeed, apparently they are friends.
Conclusion
So this is a fantastic way to upload documentation like large PDF files or a folder full of content and then ask questions related to that content. And you now have a fully functional document chatbot. I hope you enjoyed this content. Please, please like and subscribe to support my channel. And please tell me down in the comments what you would like to see me cover next. I look forward to seeing you in the next one. Bye.
