Made withLangChainClip

LangChain 101: Ask Questions On Your Custom (or Private) Files + Chat GPT

G

Greg Kamradt

Creator on the LangChain channel

Watch on YouTube ↗

122K viewsFeb 16, 202310:11 runtime

Learn how to ask questions of your own private or custom text files using LangChain and OpenAI's GPT-3. This tutorial provides a step-by-step guide on loading local documents, splitting text into manageable chunks, and creating vector embeddings with ChromaDB. Discover how to…

Chapters

#langchain#clip#aillm

Core Concept: Querying Private Data with LangChain

This video demonstrates a powerful technique for using Large Language Models (LLMs) like GPT-3 to answer questions about your private, local documents. The key is to provide the LLM with the relevant text as context, allowing it to generate answers based on information it wasn't trained on.

Important
Ensure all your Python packages (langchain, openai, chromadb, etc.) are up to date before starting, as this can prevent common errors during setup.

Step 1: Loading Documents

The first step is to load your files into the application. LangChain's document loaders make this easy.

Tool: DirectoryLoader
Usage: Point it at a folder containing your text files.
Example: loader = DirectoryLoader('./path/to/docs', glob='**/*.txt')
Function: The glob parameter ensures that only files with the .txt extension are loaded, avoiding issues with other file types that might be in the directory.

Step 2: Splitting Documents into Chunks

LLMs have context window limits, so you can't pass an entire large document at once. Documents must be split into smaller, manageable chunks.

Tool: CharacterTextSplitter
Usage: Define a chunk_size (e.g., 1000 characters) and chunk_overlap (e.g., 0).
Purpose: Breaking down long documents ensures that only the most relevant snippets of text are passed to the model, which is more efficient and fits within the context limit.

Step 3: Creating Vector Embeddings

To find the most relevant chunks for a given question, we convert the text chunks into numerical representations called embeddings and store them in a vector database.

Tool: OpenAIEmbeddings and Chroma (Vector Store)
Process:
1. The OpenAIEmbeddings service is called to convert each text chunk into a vector (a list of numbers).
2. The Chroma vector store is used to store these vectors, creating an index that can be searched efficiently.

Note
Embeddings are numerical representations of text where semantically similar pieces of text have similar vector values. This allows for fast and accurate similarity searches.

Step 4: Building the QA Chain

With the data prepared, you can build the question-answering component.

Tool: VectorDBQA from LangChain
Configuration:
- llm: The language model to use (e.g., OpenAI()).
- chain_type: The method for passing documents. 'stuff' is the simplest, as it "stuffs" all relevant chunks into a single prompt.
- vectorstore: The Chroma vector store created in the previous step.

Step 5: Running Queries and Citing Sources

Finally, you can ask questions. The chain will find the most relevant chunks from the vector store, pass them to the LLM along with your question, and return an answer.

Simple Query: qa.run("What did McCarthy discover?")

Tip
To verify where the model got its information, you can retrieve the source documents it used. This is crucial for building trust and ensuring accuracy.

Enable Sources: Set return_source_documents=True when initializing the VectorDBQA chain.

Retrieve Sources: The result object will contain a source_documents key, listing the exact text chunks used to generate the answer.

Querying Private Files with LangChain & GPT

What is going on good people? Today we have an extremely cool tutorial because we are going to learn how to take custom text files, pass them through GPT, OpenAI GPT-3, and ask questions of those text files specifically.

Now the reason why this is so cool is because we're going to use your private and local text files. This means that you can upload anything that ChatGPT can't already see, like your personal documents or notes or whatever, and you can actually go ahead and ask questions for these. Now the reason why I love this so much is because you can see how this is the starting point of some really cool capabilities. Today it's just text files that we're going to go over, but you could imagine connecting your Google Drive, your Google Slides, your text messages, your WhatsApp, whatever you may want, and it can interpret that information for you and start to give you results back. So let's jump through this example here. I have a fun one for us today.

Introducing the Dataset: Paul Graham Essays

Okay, so what we're actually going to be looking at is we're going to be looking at a series of Paul Graham essays. And if you don't know Paul Graham, he is a famous entrepreneur here in the States, and he is also an author. He has a bunch of essays, he has something like over 200. So I web scraped these and you can see that what they look like is basically just pieces of text. And so here, it's not formatted the best way, but he has a date up at the top and he has his text down below.

Now, what I want to do is I want to ask questions against these essays using ChatGPT, and I'm going to do that via the LangChain library. So you can see here we have the essays. What I went ahead and did is I just took a subset of those essays. Okay, well, it's not being nice to me. I took a subset of those essays and I just have the smallest ones right here, only because it gets kind of data intensive and heavy and it can take a while, which I don't want to do for you. So we're going to take these essays, like, well, that's a bad one. These short essays here and run them through. So let's go ahead and do that.

Setting Up: Imports and API Key

So if we take a look at our IPython notebook here, the first thing that we are going to do is we're going to import a whole bunch of things. Now LangChain has some very helpful packages that help with this. We're going to get some OpenAI embeddings. Now I'm going to go over what embeddings are in another tutorial, but think of them as a way to compress data so that we can go and search our text a lot easier later on. We're going to get Chroma, which is part of our vector store, which is where you store the embeddings. We're going to get a character splitter, which is going to split these documents up into individual chunks. We're going to get our vector database question answer from LangChain, which is the easy way to go ahead and question these things. And then we're going to get a directory loader.

Now what's really cool is in document loaders you can load a whole bunch of things, and they're adding new support every day. A directory loader is how we're going to load a directory of files. You can also do just text files if you want. We're also going to import some magic, some OS, and some NLTK for some supplemental help here.

Now I will say as I was doing this right before this video, I had a heck of a time making sure that all my packages were up to date. So if you have problems, try loading any of these sub-packages because I ran into issues for all these. And as always, make sure that your OpenAI key is set and ready to go.

Loading and Splitting Documents

So the first thing that we're going to do is we're going to instantiate our loader. loader is not defined. That is great. Let me get rid of that. Let me first load my packages here. So let's load this DirectoryLoader and where I'm pointing it at is I'm pointing it out for this IPython notebook, the relative path to the Paul Graham essay small folder. This glob thing means that you're only going to take the .txt files. Now if you're using an IPython notebook, it might throw some other files in there which is causing some problems, but that'll be your fixer for you. And then what we're going to do is we're actually going to load these documents. So here we just kind of got them ready and staged them. Here we're actually going to load them. So if I were to let's take a look at what this looks like.

If we're to load this, you can see that these are the individual essays that were written, which is cool. So it's a whole bunch of text, but this text is still too long, and we need to do something about that. So we're going to instantiate or initialize a character splitter. And a character splitter is just going to chunk up our essay into meaningful parts. Now the way Harrison described this was imagine if you had a page and there was way too many characters on it. Well, if you're going to split that page up, you might split it up by chapters. Well, if there's too many characters for the chapters, you might split it up by paragraphs. You might split it up by sentence, or you might even split it by word if you really need to go down that deep. But here we're just going to do a chunk size of 1000 and we're not going to overlap our text because that's not important for our use case here. Let's go ahead and run that.

And then so this initialized the character splitter. Now we're actually going to go do the character splitting. That was fairly... Oh, shoot. No, no, no, no, no. That's not good. That was fairly quick.

And so let's take a look at what this looks like. And it's the same thing, but these documents are going to be a whole lot smaller now because they're only going to be at the thousand character limit. So now we have individual chunks of documents that we can go ahead and then reference. Cool.

Creating Vector Embeddings with Chroma

And then the next thing that we need to do is we need to create embeddings of these. And embeddings means that you're going to change it from a string and you're going to turn it into a vector space. And that is just a fancy word of a list of values that represent the different documents inside of your documents you're looking at. And so right now we just initialized our embeddings. And now what we're actually going to... now what we're going to go do is we're going to use Chroma here and we're going to say, 'Hey, we need you to make a doc or a vector store from these texts using this embedding engine.'

So let's go ahead and run that. And now what this is doing is it's actually going and querying the OpenAI API, and there's an embeddings API. And so you can go ahead and search that settings API. OpenAI... and so you can look at embeddings here and you can read up more about it. The interesting part, well there's pricing associated to these, but you can see here what it's going to return is it's basically going to return a vector which is just a list of numbers, but you can go and check that out on your own time.

Okay cool, so it did this for us. And unfortunately, we can't see this right now. I wish we could. I wish there was an easier way to see this. If anybody finds out a way to see the actual vectors over there, let me know. Okay, cool.

Building the Question-Answering Chain

Now we're going to do is we're going to initialize our model. And so we're going to pass it our language model, which is going to be OpenAI, and we're going to do a chain type of stuff. This means what it's going to do is it's going to find the most relevant prompts from all the ones that we just broke up and it's going to stuff those into OpenAI's prompt. Now, this isn't so bad because you only make one API call and you only do the prompt there once. But imagine you had really, really long set of documents. What you may want to do is you may want to pass those to OpenAI, have them refine a little bit, and then use that output to pass it again and then refine even more and look for your answer. I'll encourage you to go look at other different other chain types on LangChain's documentation. If you want a video on that, let me know. I'm happy to step through it.

And then here's the really important part because this is the information we're going to pass our docs that we just made from Chroma, and we're going to pass those in as a vector store into here. So we'll save that into qa.

Asking Your First Question

Now, we did all that work, got our data prepped, got everything ready just to do this one thing here. So if you remember, we had our series of essays that we wanted to go and query a question against. Now I'm finally ready for a question. And I know that in the essays, there speaks about a fellow named McCarthy, who came up with the Lisp language. Now, what's really cool is I can go ahead and run this: 'What did McCarthy discover?'

And then in the background, LangChain is going to go query OpenAI with our personal, private docs we've just uploaded. And we're going to get some answers. McCarthy discovered how to use a handful of simple operators and notation for functions to build a whole programming language, which he called Lisp. That's pretty cool. So say this was even more private data that wasn't on the web anywhere, you can go ahead and use that in OpenAI to go get your own questions.

Citing Sources for Verification

Now I wanted to show you one more cool thing, which is when you can start to attribute sources to your answer. Now the way that this works is remember we split our documents into chunks. Well what LangChain will help do is it will say, 'Hey, these chunks were most helpful in getting you the answer.' And then once you have those chunks, you can start to understand why it used what it did and the information that it pulled.

So I'm going to pull the same exact function call that we did up above. The only difference is I'm going to say return_source_documents=True. And in this case, when you do that, there's a little bit of a different syntax to doing the query, but we still have the same query here. And I'm going to say using the VectorDBQA that we had above, here's the query, and I'm going to store that in a result. So it's doing the same thing. It's going to go get those.

Great. And so if I look at the result, it is not the same thing, but it's a little bit different than what we had above because there's a little bit of probabilistic outputs here. But the really cool part is when you say source_documents. And so now I just asked it for the source documents and it's telling me that, hey, these are the documents that we used in order to answer your question. Now, let me convince you about that. If I'm looking, I'm searching for McCarthy, you can see here that these different documents talk about McCarthy. Now we uploaded like 20 essays, maybe not 20, maybe 10 essays, but it knew that these were the best ones to go ahead and pick, which is pretty cool.

Conclusion and What's Next

So, that is where you can show your source documents. Now, I hope you had an awesome time with this one. Please let me know what other videos you'd like to see. But you can see how this would be the very beginning of some really cool functionality. Now in other videos, we're going to show how you can upload different data, whether it be Google Docs, whether it be Notion, whether it be Excel spreadsheets, whatever the heck you'd like in order to answer your own questions from there. Thanks, crew.