Project Introduction: A Customer Support RAG Chatbot
this project we're going to build an agent that has access to a custom knowledge base. Now this can have plenty of use cases. This could very simply be a customer facing chatbot that can answer questions about our business or we can use it internally to answer questions related to our company's data. And dealing with custom knowledge bases is one of Flowise's strengths.
Let's start by adding a new chat flow and let's call this customer support agent. In this example, I want this agent to be able to answer client's questions related to my fictitious restaurant called The Oak and Barrel. I've simply created this Word document containing all the common questions and answers related to my restaurant—things like contact information, the current specials, etc. I also created a CSV document containing all the menu items along with their prices.
So what we want to build here is an agent that has access to these custom documents. And what we could do then is simply grab this code snippet and embed it into our website, and our customers can then interact with this chatbot by clicking on this chat window and then asking questions about our business. That means we have a customer support agent that's available 24/7.
Building the Base Agent in Flowise
Let's start by adding a new node. Let's add an agent node and more specifically, we'll add the tool agent. As with the previous video, we'll simply add our chat model and for this I'll add the chat OpenAI model like so. Let's select our credentials. For the model, I'll actually just leave it on GPT-4o Mini and for the temperature, I'll lower this to 0.6.
Let's also add memory, as we want our clients to have full conversations with this agent. So under memory, let's add the buffer window memory node. We can attach it to our agent. Let's change the size to 20. Let's change the system prompt of this tool agent by clicking on additional parameters and let's change this to 'role'. And let's say: 'Your name is Max. You are a customer support agent for a restaurant called Oak and Barrel. Keep your answers precise and use the knowledge base to answer questions about the restaurant'. That's all we need for now.
Creating a Document Store and Loading Data
Now let's set up our knowledge base. Let's go back to the dashboard and let's go to Document Stores. We can use document stores to effectively create custom knowledge bases. This is a fantastic solution because we can create a document store over here and these document stores will be accessible to any of our chat flows.
Let's create a new document store. Let's call this Oak and Barrel. Let's add this. Let's open this document store and let's start attaching our knowledge sources by clicking on Add Document Loader. Here we have integration with a lot of different applications and we can upload things like CSV files. We can use the Cheerio web scraper to extract information from websites. But what I need is the DOCX file loader. So I'll upload my file, and by the way, all of the files will be available in the link in the description.
Now, if I click on 'preview chunks,' this will return all the contents in the document within a single chunk, which is not ideal. As an example, let's say the customer asks what the current specials are. We don't want to inject all of this content into the prompt when asking about specials. Instead, we only want to retrieve the most relevant text related to the specials and inject that into the prompt. So what we can do instead is split this document up into smaller chunks. We can do that by going to the text splitter and let's select the recursive character text splitter. I'll leave the chunk size on 1,000 characters with the chunk overlap of 200. Now watch what happens when we click on preview. Now we've split this document into five smaller chunks. Let's click on process.
Loading CSV Data and Preparing for Upsert
Then let's add one more additional loader. This time I'm going to use the CSV file loader to upload our menu items. For CSV documents, we actually don't have to specify a text splitter, as each row in the document will become its own chunk. So here we can see the Fillet Steak, the Ribeye Steaks, T-Bones, Sirloin Steaks, etc. Let's click on Process.
And now we have these two documents loaded into our document store. We're not done yet, though. We've simply prepared this data at this stage, and now the next step is to upsert this data into a database.
Configuring Embeddings and a Pinecone Vector Store
So what we can do is go to more actions, let's click on 'upsert all chunks'. And now we have to configure this document store. So we have to set things like the embeddings, we have to specify a vector store, and optionally we can specify a record manager. Let's start with the embeddings. Embeddings will simply take the text from the documents and convert it into vectors. It's a little bit technical to explain in this video, but the step is necessary in order for the vector store to figure out what the most relevant document to the user's question is.
So as the provider, I'm going to use OpenAI. So I'll select my credentials. For the model name, I'm going to select 'text-embedding-3-small'. We don't have to change any of these settings. The text from the documents along with these embeddings will be stored in a database called a vector store. So let's click on this, and Flowise has integration with several vector store providers. We will use Pinecone, which is free to use. So let's create our credentials. Let's give it a name like 'Pinecone API'. And for the API key, go to pinecone.io and sign up or log into your account. From here, let's create our database by clicking on 'Create Index'. Give it a name like 'flowise'. Then under Dimensions, we can manually enter the dimensions or select one of these templates. We'll use the 'text-embedding-3-small'. For the Capacity Mode, select 'serverless' and I'll leave this on AWS. Let's click on 'Create Index'.
Then let's click on API Keys. Let's create a new key. Let's call this 'flowise tutorial'. Let's create this key. Let's copy this key and paste it into Flowise. Now for the Pinecone index name, we can simply give it the index name, which we called 'flowise'. I do recommend specifying a namespace because you can reuse the same index for all of your projects. So for the namespace, I'll simply call this 'Oak and Barrel'. We don't have to change any other properties. So what we can do now is click on 'Save config and upsert'.
Verifying and Testing the Knowledge Base Retrieval
We can now see that 25 documents were added to our vector database. And if we go back to Pinecone and if we click on 'flowise', we can indeed see that 25 documents were added. And in the namespace, we can see that 'Oak and Barrel' was indeed created. And on the browser we can indeed view each of those documents that were upserted.
Another great feature about Flowise is that we can test the retrieval at this stage by clicking on 'Test Retrieval'. And let's ask 'What are the specials?' And these are the most relevant documents that were returned from our Pinecone database, and the very first document very clearly includes the specials. We can also use this screen for fine-tuning the results. So if you find that only returning four documents is not enough, we can easily change the 'top K' value to, let's say, eight documents. When we run this again, we now get eight documents back from the database. And if we feel that this actually improves the results, we can simply save these config changes. I'll simply change this back to four. Let's save the config and let's go back to our configuration.
Managing Duplicates with Record Manager
So what I do want to mention is if you do change any of this data, so let's say we add an additional document loader, you do have to run the upsert process again. But here's the issue. When I click on 'upsert', although nothing has changed, you will notice that 25 documents were added again. And if I go to our database, you will now notice that we have 50 documents. This means that all those documents were duplicated. But this is not ideal. We only want to upsert the new documents. And that is where Record Manager comes in. The record manager will keep track of the documents that we've already upserted, and when we run upsert again, it will compare our existing documents to the new set of documents and only insert the difference.
So to clear out these existing records in this database, I'm simply going to go to namespace and I'm going to delete this 'Oak and Barrel' namespace, and that will delete all the documents linked to it. So now that we have a clean start, let's add a record manager. And I highly recommend using PostGres or MySQL for this. And I do have a dedicated video taking you through the process of setting up a PostGres database and using it as a record manager. But to keep things simple for this tutorial, we'll simply add the SQLite record manager. We actually don't have to change too much. I do want to set the namespace to be the same as the namespace in our Pinecone database. And I'll change the cleanup method to 'full'. Now let's save this config. Let's run upsert, and as expected, 25 documents were added. And if we run upsert again, we can see that 25 documents were skipped this time. If we go back to our document store, we can delete entries as well. So let's delete this CSV file. Then under actions, let's click on 'upsert all chunks'. Let's click on 'upsert'. We can see that five documents were skipped, those are the documents from the Q&A doc, but all the items in the CSV file were actually deleted. So in our Pinecone database the amount of documents have been reduced to five documents. I'm just going to add the CSV file back. So I'll click on preview chunks, let's click on process. Let's run upsert again. Our menu items were added, obviously five documents were skipped. Great, we now have a custom knowledge base.
Connecting the Agent to the Knowledge Base
Next, let's give our customer agent access to this knowledge base. So back in chat flows, let's open up our customer support agent. Then under Tools, let's add the retriever tool. Let's attach this retriever tool to our agent. Let's give our tool a name like 'Oak and Barrel'. For the description, let's enter 'returns documents related to Oak and Barrel and the menu'.
Then let's go to add nodes and under vector stores let's add the document store node. We can attach the document store to this retriever tool and from the dropdown let's select 'Oak and Barrel'. If you do not see your document store in this list, it simply means the document store has not yet been upserted into your database. That should be it. Let's test this out. So in the chat let's ask 'what are your current specials?'. And indeed we get the correct response back. We can also see that the retriever tool was called, and this was the response from our vector store. In some instances you might want to site the sources of this information and show it to your users. What you can do is go to your retriever tool and enable 'Return Source Documents'. So now when we ask that same question we get the source documents as well along with their metadata.
Deploying and Embedding the Chatbot
So all that's left now is to embed this chatbot into our website by clicking on this button. Then simply copy the script and you can add it to any website simply by pasting in that script into the body tag. And if we have a look at this simple webpage, we can view this chat bubble and we can chat to our chatbot. So let's say 'hello'. Let's ask 'what are the current specials'. And we get the correct answer back, which we probably don't want to show in this case. I do want to mention that it is possible to customize the look and feel of this chat window and the chat bubble, and I have a complete video going through this process step-by-step along with instructions on embedding this chat part into different types of websites. So definitely check out that video as well.
