Made withLangChainClip

Create Your Own ChatGPT with PDF Data in 5 Minutes (LangChain Tutorial)

L

Liam Ottley

Creator on the LangChain channel

Watch on YouTube ↗

252K viewsMay 2, 20239:15 runtime

Learn to build a custom knowledge ChatGPT application using your own PDF data in just minutes with this LangChain tutorial. This guide provides a step-by-step walkthrough, from document loading and chunking to creating vector embeddings with OpenAI models and setting up a vector…

Chapters

#langchain#clip#aillm

Summary of Key Concepts

This tutorial provides a complete walkthrough for creating a custom knowledge chatbot that can answer questions about a PDF document using LangChain.

Core Architecture: Retrieval-Augmented Generation (RAG)

The system follows the RAG pattern, which involves a few key steps:

Load & Chunk: The source PDF is loaded and split into smaller, more manageable text chunks.
Embed: Each text chunk is converted into a numerical vector representation (an embedding) using an AI model.
Store: These embeddings are stored in a vector database, which is optimized for finding similar vectors.
Retrieve & Generate: When a user asks a question, their query is also embedded. The system retrieves the most relevant document chunks from the vector database and passes them, along with the original query, to a Large Language Model (LLM) to generate a final answer.

Document Processing: Chunking is Key

The quality of your chatbot's answers is highly dependent on how you process and chunk your documents.

Simple Method: Use PyPDFLoader to split the document page by page.
Advanced Method: For better results, use a more granular approach:
1. Extract raw text using a library like textract.
2. Use a RecursiveCharacterTextSplitter to create chunks of a specific size (e.g., 512 tokens) with a defined overlap (e.g., 24 tokens) to maintain context between chunks.
3. Use a tokenizer like tiktoken to accurately measure token count for chunking.

Tip
It's a best practice to visualize the distribution of your chunk sizes to ensure your chunking strategy is working as expected and you don't have chunks that are too large or small.

Vectorization and Retrieval

Embedding Model: The tutorial uses OpenAI's text-embedding-ada-002, a high-quality model for creating embeddings.
Vector Store: FAISS (Facebook AI Similarity Search) is used as an efficient, in-memory vector database. It indexes the embeddings for fast retrieval.
Similarity Search: The core retrieval mechanism is a similarity_search, which finds the document chunks whose vector embeddings are mathematically closest to the user's query vector.

Building the Chat Application

LangChain simplifies the process of tying all the components together.

Important
You must provide your own OpenAI API key for generating embeddings and getting answers from the LLM.

Basic Q&A: A simple chain can be created to take a user query, perform a similarity search to find relevant docs, and feed both into an LLM to generate a single answer.
Conversational Chatbot: To enable follow-up questions, the ConversationalRetrievalChain is used. This chain automatically manages chat history, allowing the bot to understand context from previous turns in the conversation.

Introduction to Custom Knowledge Chatbots

In this video I'm going to be showing you the fastest and easiest way that you can create a custom knowledge ChatGPT using LangChain that's trained on your own data from your own PDFs. I've seen a lot of different tutorials that have overcomplicated this a little bit so I thought I'd hop on and make a fast and to-the-point version that you're able to copy and paste my code and get started with building these custom knowledge tools for your business and for your personal use as quickly as possible.

Now if you're familiar with applications like Chat PDF where you're able to drag and drop in a document and start chatting over it, what we're going to be building today is essentially the exact same thing. You're going to be able to take that functionality and put your own PDFs in and then use it for any purposes that you like. But the best part about what I'm about to show you is that this method is going to give you complete flexibility and customization over how your app works and how the documents are processed.

Now just quickly, I'd like to plug my AI newsletter which has launched recently. Now if you want to get all of the hottest and latest AI news distilled down to a quick 5 minute read and delivered to your email, then be sure to head down below and sign up to that.

Firstly we'll be going through a very, very brief explainer on how these systems work and the different parts involved so that you can understand what we're building here and how it all works. And then secondly, we're going to be jumping straight into the notebook that I've created for this video that you're going to be able to copy and paste over to your projects and just change the name of the PDF.

How Retrieval Augmented Generation (RAG) Works

Okay guys here's a quick visualization of how this is actually working under the hood. So this is the system we are creating using LangChain, which is essentially going to take in our documents, chunk it, embed it, put it in a vector database, and then allow users to query it and get answers back. So I'll take us through the step by step now. So the first step here is to take a document and split it into smaller pieces. Now this is done because when we are recalling it and querying the database in order to get an answer based on the document, we need to receive a bunch of smaller chunks that are relevant to the user's query and not just the entire mass of information.

So step one here is to chunk it. We're going to be doing it in 512 tokens or less. So we're going to chunk our document down into however many chunks needed in order to get below those 512 tokens per piece. And then what we're going to do is take the chunks and embed each one of them one by one. So we're using the ada-002 model by OpenAI, which is by far one of the best embedding models available right now. Then we're going to be able to take all of these different embeddings for each chunk and put them into a vector database so that they're ready for recall when the user queries.

Then the final step is to allow users to actually query the database. So we do this by taking in the user's query, we put it through the exact same embedding model that we do over here, and then you query the database based on the embeddings of the user query. So we get back a number of documents that are most similar to what the user is speaking about, and then we're also able to pass that around to a large language model and include it in the context. So we take also the user's query and then the matched documents, combine them together, and ask the language model, hey can you answer this question given this context, and then we're able to send the answer back to the user. So that's a very quick high-level overview of how these applications work. Now we can jump straight into building it.

Project Setup and Loading Your PDF

At the top here we've got a summary of all the different steps we're going to be going over, so you can take a look at that but we can jump straight into these installs and imports. Now I've simplified it all down so you guys can just run these cells as you go through. So you can run that. You need to run this cell here which is going to install all the packages. My API key is already set up. You need to replace this with your API key. And once those all installed you're ready to get started.

Now for the purposes of my chat bot in this video, I'm going to be using 'Attention is All You Need', which is the Transformers research paper that was done by Google. So I thought it'd be interesting to use this within the chat bot. So here we can see I'm using it here: attention_is_all_you_need.pdf. All you need to do is come if you're using a different document, when you clone this notebook you can go over to the left side panel here and drag in your document and upload it. Once you've got it uploaded you can come back and change the name here. So replace this with the name of your PDF and then you're ready to go.

Advanced Document Chunking Strategy

The first main step we have is loading the PDFs and chunking the data with LangChain. So we've got two different methods here that I wanted to show you. One is the very easy and straightforward version that LangChain offers, which is just using this simple page loader, using PyPDFLoader. And that's just going to take the PDF that you've given it, it's going to chop it into pages, and then you're going to get all of those pages as documents ready to use in your system.

Now this method is great if you're doing a quick test, but I thought I'd show a more advanced method, which is going to be splitting up your documents into roughly similar size chunks. Now there are a number of different factors that go into creating a custom knowledge chatbot system like this and the chunk size is actually one of those and it can determine a lot in terms of the quality of the output. So this script we have here is going to allow you to split it by chunk and you can actually set the size of the chunks here. So I've got it at 512 at the moment with our overlap of 24.

Now the first step in this advanced chunking method is to use textract. And textract is going to extract all of the information out of the PDF and save it to this doc. And then second, we're going to need to save it as a text file and then reopen that text file. Now this is just to get around some issues that can frequently come up depending on the documents you use. So we save it to a new text file and then we reopen that text file. And then you need to actually create a function that allows you to count the number of tokens. So here you can see I've used a GPT-2 tokenizer and then we've just made this little function, count_tokens. This is going to take in some text in the form of a string and it's going to return the number of tokens. So this tokenizer here actually counts the number of tokens.

And then finally, we create text_splitter, which is this LangChain type called RecursiveCharacterTextSplitter. It takes in a chunk size which is variable as I mentioned, and then we need to put in the length function, which we've just created here. So final step is going to be creating the chunks objects by passing in the text that we got up here, and we've opened up from our text file, passing it into the create_documents function, and then that's going to create all of the chunks in type LangChain schema document.

Now one quick best practice that I want to show you guys is actually to do a quick visualization of the distribution of the chunks to make sure that this chunking processor has done it correctly. It's done it to the correct size that we've mentioned. So if you just run the cell, you don't need to know the specifics of it, but this essentially shows you the distribution of these different chunks. So we've got a couple that are over the limit, but that comes down to this recursive splitter. So on the most part, we don't have anything that are thousands and thousands of tokens. They're all roughly within the range that we wanted.

Building the Vector Store with FAISS

And then we need to create our vector database, which again, LangChain made super simple with this FAISS package. And we're going to take in the chunks that we created and also this embedding model, and then it's going to embed all of that, store it in the vector database, and then we're going to get this DB variable back out. Again, LangChain makes this super simple. We just need to set up our query, which is 'who created Transformers?' and then all we need to do is run a similarity search on the database using the query, and then we're going to get that back.

And there we go. So if you put this little bit in here at the bottom, which is l_docs, you can actually see that this is based on this query, it's actually pulling back four different chunks that match the query. So that's going to give you an idea of how much context is actually being grabbed from the vector database with each query.

Implementing the Q&A Chain

Then we essentially take that functionality that we've just created and combine it with a LangChain chain, which is going to take in a query. So we can do the same thing, 'who created Transformers?', we're going to retrieve the docs, and then we're going to run a chain. And that's going to take in the query and the docs, and then it's going to give us an output. So that is combining the context that's being retrieved from the similarity search with the query and then answering it as you'd expect it to.

So if we run this, 'who created Transformers?', it's going to do that similarity search, bring in the documents, then also take the user query, and then say, okay, let's run a language model on this, one of OpenAI's language models, to answer the question. And here we have the answer.

Creating an Interactive Chatbot with Memory

Now I thought I'd throw in a little extra goodie for you guys here which is to convert this functionality into an actual chat bot. So I get this a lot in my videos like, yeah, you showed us the functionality, but how can I actually use this in some kind of chatbot? So this is just a quick one that I've whipped up. If we run this, this is going to be using another LangChain component, which is going to be this conversational retrieval chain which takes in a language model and it's going to take the database that we created and use that as a retriever function. So you don't need to know too much about it, but just run the cell. And then here is a little chatbot loop that's going to allow us to interact with this knowledge base in a chat format. So here I can go, 'who created Transformers?'

And there we have it. It started to answer us. 'Were they smart?'

We have a custom knowledge chatbot using LangChain. It takes in your own PDFs, chunks them up, embeds them, creates a vector store, and then allows you to retrieve those and answer questions based on that information. And this does have chat memory included into it as you can see here. 'Who created Transformers?', gives a name. 'Were they smart?', I don't know. So here you can see that the chat memory is actually working. You have a customized chatbot with chat memory.

Conclusion and Next Steps

That about wraps it up for the video guys. Thank you so much for watching. All of this code is going to be available in the description for you to clone this notebook, change the PDF out, and start to use it for your own purposes.

Now if you've enjoyed this video and want to see more content like this, be sure to hit down below and subscribe to the channel. I'm posting tutorials like this all the time, and if you've enjoyed the video, please leave me a like. It would mean the world to me. Now as always, if this has lit up some light bulbs in your head and you want to have a chat to me as a consultant, you can book in a call with me in the description and in the pinned comment. So if you want to see some feasibility reports or talk through an idea with me, you can reach me there. And I also have my own AI development company. So if you want to build something out like this but on a bigger scale for your business or for personal use, then you can have a chat to me as a consultant and we can see if we can help you get that built.

And finally, in the description and pinned comment there are also links to join my AI entrepreneurship Discord and to sign up to my AI newsletter which is all available down there. So that's all for the video guys. Thank you so much for watching and I'll see you in the next one.